Papers with F1 score
PubHealthTab: A Public Health Table-based Dataset for Evidence-based Fact Checking (2022.findings-naacl)
Copied to clipboard
| Challenge: | Fact-checking is the task of establishing the veracity of factual information, commonly performed manually by journalists. |
| Approach: | They propose a table fact-checking dataset based on real world public health claims and noisy evidence tables from sources similar to those used by fact checkers. |
| Outcome: | The proposed dataset achieves an overall F1 score of 0.73 . |
Extraction of Texters’ Explicit Emotion Expressions in Crisis Conversations (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for extracting present and past personal emotion expressions from text-based crisis conversations are lacking in clinically relevant areas. |
| Approach: | They propose a method for extracting present and past personal emotion expressions from text-based crisis conversations and train a transformer-based model that captures contextual distinctions between true personal emotion and other mentions. |
| Outcome: | The proposed method outperforms a regex and a model trained on real conversation data and achieves an F1 score of 0.856. |
Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods (N18-2)
Copied to clipboard
| Challenge: | Existing methods for co-reference resolution focus on gender bias. |
| Approach: | They propose a new benchmark for co-reference resolution focused on gender bias, WinoBias. |
| Outcome: | The proposed system removes the bias without significantly affecting performance on existing datasets. |
Fast Prototyping a Dialogue Comprehension System for Nurse-Patient Conversations on Symptom Monitoring (N19-2)
Copied to clipboard
Zhengyuan Liu, Hazel Lim, Nur Farah Ain Suhaimi, Shao Chuen Tong, Sharon Ong, Angela Ng, Sheldon Lee, Michael R. Macdonald, Savitha Ramasamy, Pavitra Krishnaswamy, Wai Leng Chow, Nancy F. Chen
| Challenge: | a limited amount of data exists for human-human spoken dialogues for research and development . a dialogue comprehension system that extracts clinical information from spoken conversations is clinically useful . |
| Approach: | They propose a framework inspired by nurse-initiated clinical symptom monitoring conversations to construct a simulated human-human dialogue dataset. |
| Outcome: | The proposed system achieves more than 80% F1 on held-out test set from nurse-to-patient conversations. |
HateCheck: Functional Tests for Hate Speech Detection Models (2021.acl-long)
Copied to clipboard
| Challenge: | Hate speech detection models are evaluated by measuring their performance on held-out test data using metrics such as accuracy and F1 score. |
| Approach: | They propose a suite of functional tests for hate speech detection models that measure model performance on held-out test data and then craft test cases to validate their quality. |
| Outcome: | The proposed tests show that the proposed models perform poorly on a small set of widely-used hate speech datasets. |
Have my arguments been replied to? Argument Pair Extraction as Machine Reading Comprehension (2022.acl-short)
Copied to clipboard
| Challenge: | Existing studies identify argument pairs indirectly by predicting sentence-level relations between two documents, neglecting the holistic argument-level interactions. |
| Approach: | They propose to use machine reading comprehension to extract argument pairs from two documents . they propose to employ an AM query to identify all arguments in two documents, then an APE query to extract its paired arguments from another document. |
| Outcome: | The proposed method outperforms the state-of-the-art method by 7.11% in F1 score. |
A Graph Representation of Semi-structured Data for Web Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies treat semi-structured data as flat documents with pieces of text . semi-structural data is more effective to represent rich relational information . question answering is an important feature in most search engines . |
| Approach: | They propose a graph representation of Web tables and lists based on categorization of components and their relations . they also develop reasoning techniques on the graph model for the question answering task . |
| Outcome: | The proposed graph improves F1 score by 3.90 points over the state-of-the-art baselines on real datasets. |
Automatic Classification of Students on Twitter Using Simple Profile Information (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing models for age classification of students and non-students are restrictive and require access to many tweets. |
| Approach: | They propose a model which uses 3 tweet-content features to classify users as students or non-students. |
| Outcome: | The proposed model achieves an accuracy of 88.1% and an F1 score of .704 compared to previous models, which require access to many user tweets. |
SEPSIS: I Can Catch Your Lies – A New Paradigm for Deception Detection (2025.acl-srw)
Copied to clipboard
Anku Rani, Dwip Dalal, Shreya Gautam, Pankaj Gupta, Vinija Jain, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | a new framework categorizes deception into three forms: lies of omission, lies of commission, and lies of influence . a novel framework for deception detection leveraging NLP techniques is proposed . |
| Approach: | They propose a framework that categorizes deception into three forms: lies of omission, lies of commission, and lies of influence. |
| Outcome: | The proposed framework achieves an impressive F1 score of 0.87 across all layers . it can be used to investigate lies of omission, lies of commission and lies of influence . |
Language Identification and Named Entity Recognition in Hinglish Code Mixed Tweets (P18-3)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is an important text analysis task . code-mixing occurs when lexical items and grammatical features from two languages appear in one sentence . |
| Approach: | They propose to use language identifiers, parts-of-speech tags and chunkers to analyze code-mixed data. |
| Outcome: | The proposed method outperforms the best baseline by 33.18%. |
Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages (2024.naacl-srw)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a useful component in NLP applications. |
| Approach: | They propose to use annotated named entity corpora to classify a given entity into a category within a textual document. |
| Outcome: | The proposed model achieves an F1 score of 0.80 on an unseen dataset for Indian languages. |
Metaphor Detection via Explicit Basic Meanings Modelling (2023.acl-short)
Copied to clipboard
| Challenge: | Existing methods for metaphor detection use the aggregated meaning of a word to approximate its basic meaning. |
| Approach: | They propose a method which models the basic meaning of a word based on literal annotations and compares this with the contextual meaning in a target sentence to identify metaphors. |
| Outcome: | The proposed method outperforms the state-of-the-art method significantly in the F1 score and even reaches the theoretical upper bound on the VUA18 benchmark. |
Biomedical Relation Classification by single and multiple source domain adaptation (D19-62)
Copied to clipboard
| Challenge: | Existing supervised systems are highly data-driven and require a lot of effort to label data for a new domain. |
| Approach: | They propose to transfer knowledge from one or more related source domains to another domain to improve relation classification. |
| Outcome: | The proposed model outperforms neural-network based models on biomedical datasets and with contextualized embeddings on 3 biomedically-relevant datasets. |
Impact of Training Instance Selection on Domain-Specific Entity Extraction using BERT (2022.naacl-srw)
Copied to clipboard
| Challenge: | Named entity recognition (NER) tasks require a large number of training examples and handcrafted features. |
| Approach: | They propose to fine-tune pre-trained language models such as BERT to achieve up to 80% F1 when fine- tuned on only 70 training examples. |
| Outcome: | The proposed model achieves 80% F1 when fine-tuned on only 70 training examples, especially on biomedical domain. |
Unsupervised Semantic Abstractive Summarization (P18-3)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization are limited in the sense that they can never generate human level summaries for large and complicated documents. |
| Approach: | They propose a pipeline for automatic abstractive summary generation using co-reference resolution and Meta Nodes. |
| Outcome: | The proposed pipeline outperforms the state-of-the-art method by 1.7% in node prediction. |
Misspelling Detection from Noisy Product Images (2020.coling-industry)
Copied to clipboard
| Challenge: | Existing spelling research has focused on advancement in misspelling correction . a single inadvertent or intentional misspeller can propagate to large amounts of inventory . |
| Approach: | They propose a method to automatically detect misspellings from product images . they curate a large corpus and define a rich set of features to validate the method . |
| Outcome: | The proposed method improves on a large corpus and an out-of-domain public dataset and improves by 20% in the F1 score. |
AMR Alignment for Morphologically-rich and Pro-drop Languages (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing AMR aligners for English are not well suited for many languages where many concepts appear from morphologically-semantic elements. |
| Approach: | They propose to use a tree traversal approach to align AMR concepts from morphemes in a Turkish language. |
| Outcome: | The proposed aligner outperforms the existing aligners for English and Portuguese in terms of precision, recall and F1 score. |
Automated Clinical Data Extraction with Knowledge Conditioned LLMs (2025.coling-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be effective at interpreting unstructured text in reports, but they often hallucinate due to a lack of domain-specific knowledge. |
| Approach: | They propose a framework that aligns generated internal knowledge with external knowledge through in-context learning (ICL) they use a retriever to identify relevant units of internal or external knowledge and a grader to evaluate the truthfulness and usefulness of the retrieved internal-knowledge rules to align and update the knowledge bases. |
| Outcome: | Experiments with expert-curated test datasets show that the proposed framework can increase the F1 score for key fields by 12.9% over existing methods. |
RACAI’s System at PharmaCoNER 2019 (D19-57)
Copied to clipboard
| Challenge: | RACAI researchers develop named entity recognition systems for Romanian language . current system is language-independent and can be improved by using language-dependent resources . |
| Approach: | They propose to train a named entity recognition system for Romanian language . they propose to use a gazetteer-based baseline and a RNN-based NER system . |
| Outcome: | The proposed system is language independent, provided language-dependent resources exist . the proposed system can detect entities with four labels: anatomical parts, disorders, medical procedures and chemical compounds . |
Retrieval Enhanced Data Augmentation for Question Answering on Privacy Policies (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing labeled datasets are heavily imbalanced, limiting the QA performance in this domain. |
| Approach: | They propose a question answering task that captures relevant text segments from unlabeled policy documents and expands the positive examples in the training set. |
| Outcome: | The proposed framework elevates the baseline by a large margin (10% F1) and achieves a new state-of-the-art F1 score of 50%. |
Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations (D19-1)
Copied to clipboard
| Challenge: | Existing methods to analyze emotions in textual conversations are limited . emotion detection is challenging because humans rely on context and commonsense knowledge to express emotions . |
| Approach: | They propose a Knowledge-Enriched Transformer where contextual utterances are interpreted using hierarchical self-attention and external commonsense knowledge is dynamically leveraged. |
| Outcome: | The proposed model outperforms state-of-the-art models on most of the tested datasets in F1 score. |
HiPool: Modeling Long Documents Using Graph Neural Networks (2023.acl-short)
Copied to clipboard
| Challenge: | Recent work on pretraining languages have achieved satisfying results in many NLP tasks, but they are still restricted by a pre-defined maximum length. |
| Approach: | They propose a graph-based method to model sentence-level information using a fixed length and graphs to model intra- and cross-sentence correlations. |
| Outcome: | The proposed model outperforms baseline models by 2.6% in F1 score, and 4.8% on the longest sequence dataset. |
Decomposing Complex Questions Makes Multi-Hop QA Easier and More Interpretable (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Multi-hop QA requires the machine to answer complex questions through finding multiple clues and reasoning, and provide explanatory evidence to demonstrate the reasoning process. |
| Approach: | They propose a three-stage framework based on complex question decomposition that decomposes the complex question, then reads the sub-questions and then performs numerical comparison to get the final answer. |
| Outcome: | The proposed framework achieves state-of-the-art in the 2WikiMultiHopQA dataset, with a winning joint F1 score of 53.58 on the leaderboard. |
Detection of Propaganda Using Logistic Regression (D19-50)
Copied to clipboard
| Challenge: | Various propaganda techniques are used to manipulate peoples perspectives to foster a predetermined agenda. |
| Approach: | They propose a Logistic Regression-based tool that automatically classifies whether a sentence is propagandistic or not. |
| Outcome: | The proposed tool outperforms the baseline on linguistic and semantic features. |
Exploring Universal Sentence Encoders for Zero-shot Text Classification (2022.aacl-short)
Copied to clipboard
| Challenge: | Universal Sentence Encoder (USE) has gained popularity as a general-purpose sentence encoding technique. |
| Approach: | They propose to use Universal Sentence Encoder (USE) to learn a general-purpose sentence encoding technique. |
| Outcome: | The proposed technique outperforms topic-based inference in zero-shot text classification tasks. |
Exploring Statistical and Neural Models for Noun Ellipsis Detection and Resolution in English (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to noun ellipsis resolution have been sparse, using syntactic feature constraints for marking licensors and selecting their antecedents. |
| Approach: | They propose to use supervised machine learning to improve the existing F1 score by 16.55% and resolution by 14.97% for noun ellipsis subtasks. |
| Outcome: | The proposed framework improves the existing F1 score by 16.55% and the resolution subtask by 14.97%. |
A Reasoner for Real-World Event Detection: Scaling Reinforcement Learning via Adaptive Perplexity-Aware Sampling Strategy (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for abnormal event detection face two predominant limitations . existing methods rely on specialized small models and are limited by performance bottlenecks . |
| Approach: | They propose a framework that leverages the advanced reasoning capabilities of large language models for abnormal event detection. |
| Outcome: | The proposed framework achieves the highest F1 score and an average improvement of 9.59% in OOD transfer tests. |
PROMINET: Prototype-based Multi-View Network for Interpretable Email Response Prediction (2023.emnlp-industry)
Copied to clipboard
| Challenge: | a new study examines email marketing performance by considering email content and metadata. |
| Approach: | They propose a model that incorporates semantic and structural information from email data to generate latent exemplars for email response prediction. |
| Outcome: | The proposed model outperforms baseline models on two real-world email datasets . it provides interpretability through prototypes at different granularity levels while maintaining comparable performance to non-interpretable models. |
GRAVL-BERT: Graphical Visual-Linguistic Representations for Multimodal Coreference Resolution (2022.coling-1)
Copied to clipboard
Danfeng Guo, Arpit Gupta, Sanchit Agarwal, Jiun-Yu Kao, Shuyang Gao, Arijit Biswas, Chien-Wei Lin, Tagyoung Chung, Mohit Bansal
| Challenge: | Multimodal coreference resolution (MCR) is a crucial capability for building next-generation conversational agents. |
| Approach: | They propose a multimodal coreference resolution model that resolves coreferences made in multi-turn dialogues with scene images. |
| Outcome: | The proposed model resolves coreferences made in multi-turn dialogues with scene images. |
“Stupid robot, I want to speak to a human!” User Frustration Detection in Task-Oriented Dialog Systems (2025.coling-industry)
Copied to clipboard
Mireia Hernandez Caralt, Ivan Sekulic, Filip Carevic, Nghia Khau, Diana Nicoleta Popa, Bruna Guedes, Victor Guimaraes, Zeyu Yang, Andre Manso, Meghana Reddy, Paolo Rosso, Roland Mathis
| Challenge: | Detecting user frustration in task-oriented dialog systems is imperative for maintaining overall user satisfaction, engagement and retention. |
| Approach: | They compare out-of-the-box methods for user frustration detection with open-source methods . they find an LLM-based approach is promising, as it captures both emotion and dialog breakdowns . |
| Outcome: | The proposed method outperforms open-source methods in detecting user frustration in a TOD system. |
Enhancing Hyperbole and Metaphor Detection with Their Bidirectional Dynamic Interaction and Emotion Knowledge (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for hyperbole and metaphor detection focus on superficial text features, ignoring the associations of hyperbola and metaphor . Existing frameworks focus on identifying superficial text, focusing on superficial features . |
| Approach: | They propose an emotion-guided hyperbole and metaphor detection framework based on bidirectional dynamic interaction. |
| Outcome: | The proposed framework outperforms baseline methods on four datasets. |
A Controlled Reevaluation of Coreference Resolution Models (2024.lrec-main)
Copied to clipboard
| Challenge: | a pretrained language model is used in state-of-the-art coreference resolution models. |
| Approach: | They evaluate five coreference resolution models and control for language model used . they find that encoder-based CR models outperform decoder--based models in accuracy . |
| Outcome: | The encoder-based model outperforms the decoder--based models in accuracy and speed . older model generalizes the best to out-of-domain textual genres . |
Tighter Clusters, Safer Code? Improving Vulnerability Detection with Enhanced Contrastive Loss (2025.naacl-srw)
Copied to clipboard
| Challenge: | a recent study shows that supervised contrastive learning (SCL) improves embedding separation but struggles with intra-class clustering . code vulnerability detection is a cornerstone of software security, especially as the world undergoes rapid digitization. |
| Approach: | They propose an extension of SCL with a distance-based regularization term that tightens intra-class clustering while maintaining inter-class separation. |
| Outcome: | The proposed method improves F1 score on CodeBERT and GraphCodeBERT with BCE, BCE + SCL, and BCE+ CESCL. |
A Corpus and Method for Chinese Named Entity Recognition in Manufacturing (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing resources and techniques for named entity recognition (NER) for manufacturing-specific named entities are limited. |
| Approach: | They propose a corpus of Chinese manufacturing specifications, named MS-NERC, with 4,424 sentences and 16,383 entities. |
| Outcome: | The proposed model outperforms neural methods in few-shot and rich-resource domains. |
Noise Robust Named Entity Understanding for Voice Assistants (2021.naacl-industry)
Copied to clipboard
Deepak Muralidharan, Joel Ruben Antony Moniz, Sida Gao, Xiao Yang, Justine Kao, Stephen Pulman, Atish Kothari, Ray Shen, Yinying Pan, Vivek Kaul, Mubarak Seyed Ibrahim, Gang Xiang, Nan Dun, Yidan Zhou, Andy O, Yuan Zhang, Pooja Chitkara, Xuan Wang, Alkesh Patel, Kushal Tayal, Roger Zheng, Peter Grasch, Jason D Williams, Lin Li
| Challenge: | Named Entity Recognition and Entity Linking are challenging for voice assistants . utterances are relatively short, so there is not much context to help disambiguate . |
| Approach: | They propose a Named Entity Understanding system that combines NER and EL in a joint reranking module. |
| Outcome: | The proposed framework improves NER accuracy by up to 3.13% and EL accuracy by 3.6% in F1 score . it also leads to better accuracies in other natural language understanding tasks . |
MATSA: Multi-Agent Table Structure Attribution (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Tabular data present unique challenges for attribution due to ambiguities, complex header hierarchies, and the difficulty in interpreting individual table cells without row and column context. |
| Approach: | They propose a task to generate row and column-level attributions supporting LLM-generated answers. |
| Outcome: | The proposed task outperforms baselines on tabCite and improves F1 score. |
MedTem2.0: Prompt-based Temporal Classification of Treatment Events from Discharge Summaries (2023.acl-srw)
Copied to clipboard
| Challenge: | Clinical texts contain important temporal information, such as medication start and end dates, appointment dates, and diagnosis dates. |
| Approach: | They propose to use prompt-based learning and fine-tuning to classify temporal relations between treatments and hospitalisation periods in discharge summaries. |
| Outcome: | The proposed method identifies whether a treatment was administered between the time of admission and discharge from the hospital. |
Exploring the Robustness of Task-oriented Dialogue Systems for Colloquial German Varieties (2024.eacl-long)
Copied to clipboard
| Challenge: | Mainstream cross-lingual task-oriented dialogue systems often overlook the transfer to lower-resource colloquial varieties due to limited test data. |
| Approach: | They propose to train a model for intent recognition and slot-filling in English and apply it to other languages. |
| Outcome: | The proposed model performs better than existing models on English and other languages. |
Multi-Input Multi-Output Sequence Labeling for Joint Extraction of Fact and Condition Tuples from Scientific Text (D19-1)
Copied to clipboard
| Challenge: | Existing methods to extract factual tuples from scientific text do not consider conditions. |
| Approach: | They propose a new sequence labeling framework to jointly extract fact and condition tuples from scientific sentences. |
| Outcome: | The proposed framework improves F1 score relative to existing methods by 4.2% and 6.2% on bioNLP2013. |
Pretrained Knowledge Base Embeddings for improved Sentential Relation Extraction (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing models that perform explicit on-task training of graph embeddings are inadequate. |
| Approach: | They propose to combine pretrained knowledge base graph embeddings with transformer based language models to improve performance on sentential Relation Extraction task. |
| Outcome: | The proposed model outperforms state-of-the-art models on the sentential Relation Extraction task. |
DuRE: Dual Contrastive Self Training for Semi-Supervised Relation Extraction (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing document-level relation extraction methods require manual training and labeled data to obtain supervised learning. |
| Approach: | They propose a document-level relation extraction framework that integrates RE and text generation as a dual process. |
| Outcome: | The proposed framework significantly boosts recall and F1 score with comparable precision on two document-level RE tasks against several strong baselines. |
Learning with Partial Annotations for Event Detection (2023.acl-long)
Copied to clipboard
| Challenge: | Event detection (ED) requires fully labeled and high-quality training data. |
| Approach: | They propose a new trigger localization formulation using contrastive learning to distinguish ground-truth triggers from contexts and show a decent robustness for addressing partial annotation noise. |
| Outcome: | The proposed approach achieves an F1 score of over 60% in an extreme scenario where 90% of events are unlabeled. |
MDC3: A Novel Multimodal Dataset for Commercial Content Classification in Bengali (2025.naacl-srw)
Copied to clipboard
| Challenge: | Identifying commercial posts in resource-constrained languages remains a challenge for automatic text classification tasks. |
| Approach: | They propose a dataset for Bengali social media posts classified as commercial and noncommercial . they include an annotation guideline to aid future dataset creation in resource-constrained languages . |
| Outcome: | The proposed dataset is based on an annotation guideline for future dataset creation in resource-constrained languages. |
SOM-NCSCM : An Efficient Neural Chinese Sentence Compression Model Enhanced with Self-Organizing Map (2021.emnlp-main)
Copied to clipboard
| Challenge: | Sentence Compression (SC) is an important natural language processing task . it aims to shorten sentences while preserving the original meanings of the words . improvements on Chinese SC models are still lacking due to several difficulties . |
| Approach: | They propose a neural Chinese SC model enhanced with a Self-Organizing Map from Chinese colloquial sentences from a real-life question answering system. |
| Outcome: | The proposed model achieves a promising F1 score of 89.655 and BLEU4 score of 70.116 . it improves the performance of the whole neural Chinese SC model in a valid manner . |
DocEE-zh: A Fine-grained Benchmark for Chinese Document-level Event Extraction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Chinese document-level event extraction is still largely unexplored. |
| Approach: | They propose a Chinese document-level event extraction dataset with over 36,000 events and 210,000 arguments. |
| Outcome: | The proposed dataset includes over 36,000 events and more than 210,000 arguments . it is an extension of the DocEE dataset, utilizing the same event schema and annotated by human experts. |
Efficient and Accurate Prompt Optimization: the Benefit of Memory in Exemplar-Guided Reflection (2025.acl-long)
Copied to clipboard
Cilin Yan, Jingyun Wang, Lin Zhang, Ruihui Zhao, Xiaopu Wu, Kai Xiong, Qingsong Liu, Guoliang Kang, Yangyang Kang
| Challenge: | Recent work utilizes feedbacks generated from erroneous cases to guide prompt optimization . previous methods rely on computational resources and powerful GPUs . |
| Approach: | They propose an automatic prompt engineering method that leverages feedbacks from erroneous cases to guide prompt optimization. |
| Outcome: | The proposed method surpasses state-of-the-art methods with less steps and lower computational resources. |
Building a Corpus from Handwritten Picture Postcards: Transcription, Annotation and Part-of-Speech Tagging (L18-1)
Copied to clipboard
| Challenge: | In this paper, we describe the processes and challenges of digitalisation, manual transcription, and manual annotation of over 11,000 postcards. |
| Approach: | They describe the processes and challenges of digitalisation, manual transcription, and manual annotation of over 11,000 postcards written in German and Swiss German. |
| Outcome: | The proposed system outperforms state-of-the-art taggers in the evaluation of the 'picture postcard corpus' containing over 11,000 handwritten postcards . |
XFormParser: A Simple and Effective Multimodal Multilingual Semi-structured Form Parser (2025.coling-main)
Copied to clipboard
Xianfu Cheng, Hang Zhang, Jian Yang, Xiang Li, Weixiao Zhou, Fei Liu, Kui Wu, Xiangyuan Guan, Tao Sun, Xianjie Wu, Tongliang Li, Zhoujun Li
| Challenge: | Document AI parsing semi-structured image form is a key information extraction task. |
| Approach: | They propose a multimodal and multilingual semi-structured FORM PARSER which integrates SER and relation extraction into a unified framework. |
| Outcome: | The proposed framework achieves up to 1.79% improvement on RE tasks in multilingual and zero-shot settings. |
Efficient Diagnosis Assignment Using Unstructured Clinical Notes (2023.acl-short)
Copied to clipboard
| Challenge: | Electronic phenotyping entails using electronic health records (EHRs) to identify patients with specific clinical outcomes and determine when those outcomes occurred. |
| Approach: | They propose a framework for electronic phenotyping that integrates labeling functions and a disease-agnostic neural network to assign diagnoses to patients. |
| Outcome: | The proposed framework disambiguates hypertension true positives and false positives with a supervised area under the precision-recall curve (AUPRC) of 0.85. |
Fact Checking with Insufficient Evidence (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing work on how to automate fact checking relies on information obtained from external sources. |
| Approach: | They propose a fluency-preserving method for omitting information from the evidence at the constituent and sentence level and a diagnostic dataset for FC with omitted evidence. |
| Outcome: | The proposed method improves evidence sufficiency prediction by 17.8 F1 score and 2.6 F1 scores. |
Error Detection in Large-Scale Natural Language Understanding Systems Using Transformer Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Large-scale conversational assistants process every utterance using multiple models for domain, intent and named entity recognition. |
| Approach: | They combine utterance encodings from a RoBERTa model with the Nbest hypothesis produced by the production system to detect domain classification errors. |
| Outcome: | The proposed approach outperforms bi-LSTM models and a standalone model by 2.2% to 32.2% by ensembling multiple models. |
CodeGuard: Improving LLM Guardrails in CS Education (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly embedded in Computer Science classrooms to automate code generation, feedback, and assessment. |
| Approach: | They propose a guardrail framework for educational AI systems that can handle unsafe and irrelevant prompts. |
| Outcome: | The proposed framework reduces potentially harmful or policy-violating code completions by 30-65% without degrading performance on legitimate educational tasks. |
Deep Enhanced Representation for Implicit Discourse Relation Recognition (C18-1)
Copied to clipboard
| Challenge: | Discourse parsing requires understanding of text spans and can't be easily derived from surface features from sentence pairs. |
| Approach: | They propose a model augmented with different grained text representations to improve discourse relation recognition. |
| Outcome: | The proposed model achieves state-of-the-art accuracy with greater than 48% in 11-way and F1 score greater than 50% in 4-way classifications for the first time according to our best knowledge. |
The Fallacy of Echo Chambers: Analyzing the Political Slants of User-Generated News Comments in Korean Media (D19-55)
Copied to clipboard
| Challenge: | a new study analyzes the political slants of user comments on partisan media in Korea . the classifiers detect political leaning on conservative and liberal news outlets . |
| Approach: | They built a BERT-based classifier to detect political leaning of short comments . they found a high presence of conservative bias on conservative and liberal news outlets . |
| Outcome: | The proposed classifier produced an F1 score of 0.83 for 21.6K comments . it shows that more liberals comment on stories resonating with their political perspectives . |
Improving Few-Shot Cross-Domain Named Entity Recognition by Instruction Tuning a Word-Embedding based Retrieval Augmented Large Language Model (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to named entity recognition are domain specific and require a domain specific architecture. |
| Approach: | They propose a retrieval augmented large language model for Named Entity Recognition . the model uses word-embedding over sentence-level embedding to fine tune . |
| Outcome: | The proposed model outperforms existing models on the CrossNER dataset. |
Incremental Event Detection via Knowledge Consolidation Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to event detection require a fixed set of pre-defined event types . existing methods cannot handle semantic ambiguity and training data imbalance problems . |
| Approach: | They propose a Knowledge Consolidation Network to address these issues . they propose to use a prototype enhanced retrospection and hierarchical distillation to mitigate the adverse effects of semantic ambiguity and class imbalance. |
| Outcome: | The proposed method outperforms the state-of-the-art model by 19% and 13.4% of whole F1 score on ACE and TAC benchmarks. |
InsightNet : Structured Insight Mining from Customer Feedback (2023.emnlp-industry)
Copied to clipboard
Sandeep Sricharan Mukku, Manan Soni, Chetan Aggarwal, Jitenkumar Rana, Promod Yenigalla, Rashmi Patange, Shyam Mohan
| Challenge: | Existing methods for extracting structured insights from reviews suffer from drawbacks . lack of structure, non-standard aspect names, lack of abundant training data limit their effectiveness and applicability. |
| Approach: | They propose a semi-supervised multi-level taxonomy from raw customer reviews and a semantic similarity heuristic approach to generate labelled data. |
| Outcome: | The proposed approach outperforms existing methods in structure, hierarchy and completeness. |
Named Entity Recognition for Chinese biomedical patents (2020.coling-main)
Copied to clipboard
| Challenge: | Existing attempts to address NER for Chinese biomedical texts have been limited due to the amount of Chinese biomedicine discoveries being patented. |
| Approach: | They train and evaluate Chinese biomedical patents NER models based on BERT . their model is optimized for Chinese bio-patent data and scored an F1 . |
| Outcome: | The proposed model achieves an F1 score of 0.540.15 for Chinese biomedical patent data. |
Deploying Unified BERT Moderation Model for E-Commerce Reviews (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Product reviews and ratings are an integral part of the shopping experience to build trust among users. |
| Approach: | They propose a system and a model for automated moderation of user-generated content that can be pre-trained with monolingual data to cope with the domain gap. |
| Outcome: | The proposed model achieves 4.78% increase in F1 score with less labelled data and 2.57% increase in score on the review data compared to the publicly available models. |
TIMBERT: Toponym Identifier For The Medical Domain Based on BERT (2020.coling-main)
Copied to clipboard
| Challenge: | Phylogeographers have long linked the geographical spread of viruses to the growth in global tourism and international trade of goods. |
| Approach: | They propose to automate the process of place name detection in the medical domain by learning the mapping from an input sentence to the associated sentence labeled with toponyms. |
| Outcome: | The proposed model achieves an F1 score of 90.85%, compared to the state-of-the-art of 89.13%. |
End-to-End Speech to Intent Prediction to improve E-commerce Customer Support Voicebot in Hindi and English (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Experimental results show that an E2E S2I model outperforms a conventional pipeline by 27% on the F1 score . multi-component pipelines require large annotated datasets, have higher latency, and have complex deployment. |
| Approach: | They propose an end-to-end (E2E) S2I model that uses pre-trained automatic speech recognition with slight modification and fine-tuning on small annotated datasets. |
| Outcome: | The proposed model outperforms a conventional pipeline on an annotated dataset by 27% on the F1 score. |
Structure Regularized Neural Network for Entity Relation Classification for Chinese Literature Text (N18-2)
Copied to clipboard
| Challenge: | Existing methods for relation classification have been used in natural language processing. |
| Approach: | They propose a relation classification task for Chinese literature text using a new dataset. |
| Outcome: | The proposed model outperforms the state-of-the-art methods on Chinese literature text. |
Zero-shot Slot Filling in the Age of LLMs for Dialogue Systems (2025.coling-industry)
Copied to clipboard
| Challenge: | Existing methods for zero-shot slot filling focus on text data, overlooking conversational data. |
| Approach: | They propose a method for automatic data annotation with slot induction and black-box knowledge distillation from a teacher LLM to a smaller model. |
| Outcome: | The proposed method outperforms existing models on internal datasets by 26% relative increase in F1 score. |
Filter-And-Refine: A MLLM Based Cascade System for Industrial-Scale Video Content Moderation (2025.acl-industry)
Copied to clipboard
Zixuan Wang, Jinghao Shi, Hanzhong Liang, Xiang Shen, Vera Wen, Zhiqian Chen, Yifan Wu, Zhixin Zhang, Hongyu Xiong
| Challenge: | Effective content moderation is essential for video platforms to safeguard user experience and uphold community standards. |
| Approach: | They propose a method to transform a generative MLLM into a multimodal classifier using minimal discriminative training data. |
| Outcome: | The proposed method improves F1 score by 66.50% over traditional classifiers while requiring only 2% of the fine-tuning data. |
MIMICause: Representation and automatic extraction of causal relation types from clinical notes (2022.findings-acl)
Copied to clipboard
| Challenge: | Extracted causal information from clinical notes can be combined with structured EHR data such as demographics, diagnoses, and medications. |
| Approach: | They propose to annotate clinical notes and develop an annotated corpus and provide baseline scores to identify types and direction of causal relations between a pair of biomedical concepts. |
| Outcome: | The proposed annotation guidelines achieved a high inter-annotator agreement and a macro F1 score on the clinical text. |
LNN-EL: A Neuro-Symbolic Approach to Short-text Entity Linking (2021.acl-long)
Copied to clipboard
Hang Jiang, Sairam Gurajada, Qiuhao Lu, Sumit Neelam, Lucian Popa, Prithviraj Sen, Yunyao Li, Alexander Gray
| Challenge: | Existing work deals with EL in the context of longer text, such as a sentence. |
| Approach: | They propose a neuro-symbolic approach that uses interpretable rules based on first-order logic to achieve better performance with black-box neural approaches. |
| Outcome: | The proposed approach achieves better performance than heuristics-based approaches on short-text EL . it can easily blend existing rule templates with multiple types of features, and even with scores resulting from previous EL methods. |
Linguistically Conditioned Semantic Textual Similarity (2024.acl-long)
Copied to clipboard
| Challenge: | Semantic textual similarity (STS) is a fundamental NLP task that measures the semantic similarity between two sentences. |
| Approach: | They propose to use a conditional STS dataset to measure sentences’ similarity conditioned on a certain aspect to reduce the inherent ambiguity posed by the sentences. |
| Outcome: | The proposed method improves the performance over baselines on the C-STS dataset with over 80% F1 score. |
AnnoHID: LLM-Assisted Annotation Framework for Low-Resource Medical Texts (2026.acl-demo)
Copied to clipboard
Annisa Maulida Ningtyas, Guntur Budi Herwanto, Yunita Sari, Rifki Afina Putri, Filip Kovacevic, Alaa El-Ebshihy, Varvara Arzt, Florina Piroi
| Challenge: | Social media platforms are a popular way to communicate with medical experts and improve health literacy. |
| Approach: | They introduce a semi-automated annotation framework for medical texts in low-resource languages . they use large language models for pre-annotation and human validation to support efficient annotation . |
| Outcome: | The proposed framework is applied to medical social media texts in Bahasa Indonesia . it yields higher inter-annotator agreement and human review improves output . future work focuses on mitigating pre-annotation bias and reducing annotation overhead . |
Neural Document Segmentation Using Weighted Sliding Windows with Transformer Encoders (2025.coling-industry)
Copied to clipboard
| Challenge: | Using overlapping text sequences and position-aware weighting, we achieve up to a 10% increase in segmentation F1 score compared to existing methods. |
| Approach: | They propose a Transformer-based method for document segmentation that utilizes overlapping text sequences with a unique position-aware weighting mechanism to enhance segmentation accuracy. |
| Outcome: | The proposed method achieves up to 10% increase in segmentation F1 score compared to existing methods and improves quality of generated responses by 5% while achieving four times greater efficiency. |
KnowledgeNet: A Benchmark Dataset for Knowledge Base Population (D19-1)
Copied to clipboard
| Challenge: | KnowledgeNet provides text exhaustively annotated with facts . high-quality KBs still rely almost exclusively on human-curated structured or semi-structured data. |
| Approach: | They propose five baseline approaches to populating a knowledge base with facts . the best approach achieves an F1 score of 0.50, significantly outperforming a traditional approach by 79% . |
| Outcome: | The best approach achieves an F1 score of 0.50, outperforming a traditional approach by 79%, indicating the dataset is challenging. |
Intent-calibrated Self-training for Answer Selection in Open-domain Dialogues (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing answer selection models require large amounts of labeled data to produce accurate answers. |
| Approach: | They propose intent-calibrated self-training to calibrate answer labels using labeled data . they propose intentcalibration to improve quality of pseudo answer labels . |
| Outcome: | The proposed intent-calibrated answer selection paradigm outperforms baselines with 1%, 5%, and 10% labeled data on two benchmark datasets. |
Reviving Iterative Refinement in Diffusion-based NER with an Initializer-Restorer Approach (2026.acl-short)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a fundamental task in Information Extraction. |
| Approach: | They propose a generative paradigm for Named Entity Recognition by modeling NER as a boundary diffusion process. |
| Outcome: | The proposed model performs better than baseline on ACE2004, GENIA, and CleanCoNLL. |
Meta Self-Refinement for Robust Learning with Weak Supervision (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent methods leverage self-training to build noise-resistant models . however, the teacher trained under weak supervision may have fitted a substantial amount of noise and therefore produce incorrect pseudo-labels. |
| Approach: | They propose a framework that encourages teacher to refine its pseudo-labels to effectively combat label noise from weak supervision. |
| Outcome: | The proposed framework outperforms state-of-the-art methods by 11.4% in accuracy and 9.26% in F1 score on eight NLP benchmarks. |
Improving Multimodal Classification of Social Media Posts by Leveraging Image-Text Auxiliary Tasks (2024.findings-eacl)
Copied to clipboard
| Challenge: | Prior work on multimodal content classification has not addressed these challenges. |
| Approach: | They propose to use two auxiliary tasks to fine-tune multimodal models to address hidden cross-modal semantics and weak image-text relationships when modeling text and images. |
| Outcome: | The proposed model improves by up to 2.6 F1 score across five diverse social media datasets. |
Sequential Span Classification with Neural Semi-Markov CRFs for Biomedical Abstracts (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for dividing biomedical abstracts into rhetorical segments assign a rhetorical label to each sentence while considering context in the abstract. |
| Approach: | They propose to use Neural Semi-Markov Conditional Random Fields to assign a rhetorical label to a span that consists of continuous sentences. |
| Outcome: | The proposed method achieved the best micro sentence-F1 score and the best macro span-F1. |
VlogQA: Task, Dataset, and Baseline Models for Vietnamese Spoken-Based Machine Reading Comprehension (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing datasets for machine reading comprehension tasks in Vietnamese focus on written documents, such as Wikipedia articles, online newspapers, or textbooks. |
| Approach: | They propose to capture Vietnamese spoken language in natural settings and use it to create a machine-learning corpus for machine reading comprehension tasks. |
| Outcome: | The proposed corpus consists of 10,076 question-answer pairs based on 1,230 transcript documents sourced from YouTube . |
MixRevDetect: Towards Detecting AI-Generated Content in Hybrid Peer Reviews. (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing methods for detecting fully AI-generated peer reviews fail to detect finer-grained AI-generated points within mixed-authorship reviews. |
| Approach: | They propose a method to identify AI-generated points in peer reviews using large language models . their approach achieved an F1 score of 88.86%, significantly outperforming existing methods . |
| Outcome: | The proposed method outperforms existing methods in identifying AI-generated points in peer reviews. |
Prosodic segmentation for parsing spoken dialogue (2021.acl-long)
Copied to clipboard
| Challenge: | Existing parsers struggle to parse spoken dialogue because of disfluencies and unmarked boundaries between sentence-like units (SUs). |
| Approach: | They hypothesize that prosody affects a parser that receives an entire dialogue turn as input, instead of gold standard pre-segmented SUs. |
| Outcome: | The proposed model performs better than the SU-based model on the English Switchboard corpus despite performing two tasks rather than one, and pitch and intensity features are the most important for this corpus. |
FaithDial: A Faithful Benchmark for Information-Seeking Dialogue (2022.tacl-1)
Copied to clipboard
| Challenge: | a new benchmark for hallucination-free dialogues is based on knowledge-based conversational models that generate unsupported utterances . a recent study shows that models that are trustworthy generate unverifiable or factually incorrect statements . |
| Approach: | They propose a data-centric solution to edit hallucinated responses in the Wizard of Wikipedia benchmark. |
| Outcome: | The proposed model improves on the Wizard of Wikipedia benchmark while maintaining engaging conversations. |
Learning to Generalize for Cross-domain QA (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for QA are hampered by increased training costs . current methods suffer significant performance degradation when applied to out-of-domain examples. |
| Approach: | They propose a method that combines prompting methods and linear probing with fine-tuning strategy, which does not entail additional cost. |
| Outcome: | The proposed method outperforms state-of-the-art baselines with an average increase in F1 score of 4.5%-7.9%. |
Simple and Effective Semi-Supervised Question Answering (N18-2)
Copied to clipboard
| Challenge: | Existing deep learning systems for extractive Question Answering are limited and expensive to construct. |
| Approach: | They propose a semi-supervised QA system where end user specifies a set of documents and only a few labelled examples. |
| Outcome: | The proposed system achieves 50% F1 score on SQuAD and TriviaQA with very little labeled data. |
Zero-Shot Information Extraction as a Unified Text-to-Triple Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a number of information extraction tasks require task-specific training. |
| Approach: | They propose a text-to-triple translation framework for information extraction tasks . they propose enabling task-agnostic translation by leveraging latent knowledge of a pre-trained language model . |
| Outcome: | The proposed framework outperforms the existing methods on open information extraction tasks. |
MIDAS: A Dialog Act Annotation Scheme for Open Domain HumanMachine Spoken Conversations (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing dialog act schemes are designed for human-human conversations, but are not suitable for automatic speech recognition. |
| Approach: | They propose a dialog act annotation scheme for open-domain human-machine conversations . they collected 24K utterances from a large open- domain spoken conversation dataset . |
| Outcome: | The proposed scheme achieves an F1 score of 0.79 on a 24K spoken conversation dataset. |
Beyond Guilt: Legal Judgment Prediction with Trichotomous Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current legal large language models lack trichotomous reasoning capabilities due to the absence of an appropriate benchmark dataset. |
| Approach: | They propose a benchmark dataset for Legal Judgment Prediction with Innocent Verdicts that incorporates trichotomous dogmatics into zero-shot prompting and fine-tuning. |
| Outcome: | The proposed dataset extends three widely-used legal datasets through LLM-based augmentation and manual verification. |
TableMBR: Minimum Bayes Risk Table Generation Based on Structural Consistency (2026.acl-srw)
Copied to clipboard
| Challenge: | Experimental results show TableMBR outperforms the baseline, achieving relative improvements of up to 15% in F1 on Rotowire and 23% in accuracy on LiveSum. |
| Approach: | They propose a text-to-table task that generates structured data from unstructured text . they propose 'tableMBR' that maintains structural consistency through minimum Bayes risk decoding . |
| Outcome: | The proposed method outperforms the baseline and achieves relative improvements in F1 and LiveSum. |
Utilizing Character and Word Embeddings for Text Normalization with Sequence-to-Sequence Models (D18-1)
Copied to clipboard
| Challenge: | Recent advances in text normalization have limited applications in other languages . a novel approach to text normalizing uses character embeddings and word embedds . |
| Approach: | They propose a sequence-to-sequence model with character-based attention that uses pre-trained word embeddings to model subword information. |
| Outcome: | The proposed model achieves state-of-the-art F1 score on Arabic spelling correction task despite being small and unsuited for the task. |
R-VGAE: Relational-variational Graph Autoencoder for Unsupervised Prerequisite Chain Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Concept prerequisite chain learning is an unsupervised task with no access to labeled concept pairs during training. |
| Approach: | They propose a model that uses deep learning representations to predict concept relations . they frame concept prerequisite chain learning as an unsupervised task with no labeled concept pairs . |
| Outcome: | The proposed model outperforms semi-supervised methods in terms of accuracy and F1 score. |
Towards Generalized Open Information Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Open Information Extraction (OpenIE) models are evaluated on in-domain test sets aside from the training corpus, which violates the initial task principle of domain-independence. |
| Approach: | They propose to generalize OpenIE over unseen target domains with different data distributions from source training domains. |
| Outcome: | The proposed method beats the previous methods in both in- and out-of-domain settings by 6.0% in F1 score absolutely. |
K-LegalDeID: A Benchmark Dataset and KLUEBERT-CRF for De-identification in Korean Court Judgments (2026.eacl-long)
Copied to clipboard
| Challenge: | Korean legal system mandates public access to court judgments to ensure judicial transparency, but this requirement conflicts with privacy protection obligations due to the prevalence of Personally Identifiable Information (PII) in legal documents. |
| Approach: | They propose a large-scale benchmark dataset and an efficient KLUEBERT-CRF model for de-identification for Korean court judgments. |
| Outcome: | The proposed model achieves state-of-the-art performance with an entity-level micro F1 score of 0.9923. |
Enhancing Marker Scoring Accuracy through Ordinal Confidence Modelling in Educational Assessments (2025.acl-industry)
Copied to clipboard
| Challenge: | Automated Essay Scoring (AES) systems aim to evaluate the quality of candidate writing using computational methods. |
| Approach: | They propose a model that assigns a confidence score to each automated score to ensure it meets high reliability standards. |
| Outcome: | The proposed model achieves an F1 score of 0.97 and releases 47% of predicted scores with 100% CEFR agreement and 99% with at least 95% CEFR agreeance compared to the standalone model where all predicted scores are released. |
Hierarchical Reason-of-Contact Detection in Retail Banking Customer Interactions via LLM-Driven Taxonomy Induction (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing approaches to define customer intents or contact reasons are fragmented and manual . existing systems fail to capture the linguistic diversity of thousands of daily callers . |
| Approach: | They propose a framework that develops a hierarchical Reason-of-Contact taxonomy . it covers hundreds of business processes and can be deployed in real time . |
| Outcome: | The proposed framework achieves 10% improvement in F1 score over baseline approaches on a reference dataset. |
Beyond Silent Letters: Amplifying LLMs in Emotion Recognition with Vocal Nuances (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that Large Language Models possess a form of emotional intelligence, capable of interpreting emotional stimuli in text. |
| Approach: | They propose a method that translates speech characteristics into natural language descriptions and integrates them into LLMs to perform multimodal emotion analysis via text prompts. |
| Outcome: | The proposed method outperforms baseline models that require structural modifications on two datasets showing significant improvements in emotion recognition accuracy. |
Robustness Evaluation of the German Extractive Question Answering Task (2025.coling-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for Question Answering systems only include EM and F1 scores, but they overlook critical factors for the deployment of QA systems. |
| Approach: | They propose to define an evaluation method specifically tailored to the German language to evaluate the robustness of German QA models. |
| Outcome: | The proposed method extends existing methods to German language . it shows that all models are vulnerable to character-level perturbations . |
Towards Robust Extractive Question Answering Models: Rethinking the Training Methodology (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models lack robustness against distribution shifts and adversarial attacks when training on unanswerable questions in EQA datasets. |
| Approach: | They propose a novel loss function for the EQA problem to improve the robustness of extractive question answering models by adding adversarial questions to a crowdsourcing process. |
| Outcome: | The proposed method maintains in-domain performance while improving on out-of-domain datasets. |
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have shown remarkable results on various NLP tasks. |
| Approach: | They propose to improve the feed-forward network (FFN) in BERT with a higher computational cost than improving the multi-head attention (MHA). |
| Outcome: | The proposed model is 6.9 smaller and 4.4 faster than BERTBASE and has competitive performances on GLUE and SQuAD Benchmarks. |
Exploiting Contextual Information via Dynamic Memory Network for Event Detection (D18-1)
Copied to clipboard
| Challenge: | Existing methods for event detection only process context once . a multi-hop mechanism to capture contextual information improves performance . |
| Approach: | They propose to use dynamic memory network to capture contextual information . they propose to model event triggers by identifying word or phrase which most represents it . |
| Outcome: | The proposed model achieves best F1 score compared to the state-of-the-art models. |
HalluDetect: Detecting, Mitigating, and Benchmarking Hallucinations in Conversational Systems in the Legal Domain (2025.emnlp-industry)
Copied to clipboard
Spandan Anaokar, Shrey Ganatra, Swapnil Bhattacharyya, Harshvivek Kashid, Shruthi N Nair, Reshma Sekhar, Siddharth Manohar, Rahul Hemrajani, Pushpak Bhattacharyya
| Challenge: | Large Language Models (LLMs) are widely used in industry but still produce hallucinations, limiting their reliability in critical applications. |
| Approach: | They propose to reduce hallucinations in consumer grievance chatbots by reducing their token accuracy by 0.4159 per turn. |
| Outcome: | The proposed system achieves an F1 score of 68.92% outperforming baseline detectors by 22.47% while maintaining the highest token accuracy. |
Development of a Benchmark Corpus to Support Entity Recognition in Job Descriptions (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing tools for identifying and extracting salient entities from job descriptions are limited by the lack of publicly available training data. |
| Approach: | They propose to use a standard definition of entities and a training corpus to develop a benchmark Entity Recognition (ER) model. |
| Outcome: | The proposed model achieves an F1 score of 0.59 from 18.6k entities comprising five types (Skill, Qualification, Experience, Occupation, and Domain). |
A Unified Framework for N-ary Property Information Extraction in Materials Science (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a framework for extracting n-ary property information from materials science literature is proposed . the framework addresses the critical challenge of capturing complex relationships that span multiple sentences. |
| Approach: | They propose a framework for extracting n-ary property information from materials science literature . they propose three complementary approaches to capture complex relationships that span multiple sentences . |
| Outcome: | The proposed framework outperforms existing methods in n-ary property extraction tasks. |
Teaching Neural Module Networks to Do Arithmetic (2022.coling-1)
Copied to clipboard
| Challenge: | Neural Module Networks (NMNs) have limited reasoning abilities and lack numerical reasoning capability. |
| Approach: | They propose to integrate the original question in the interpreter and introduce addition and subtraction modules that perform numerical reasoning over numbers. |
| Outcome: | The proposed methods outperform previous state-of-the-art models on a subset of DROP and achieve competitive reasoning performance. |
Pay More Attention to Relation Exploration for Knowledge Base Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches focus on entity representation and final answer reasoning, which results in limited supervision for this task. |
| Approach: | They propose a framework that utilizes relations to enhance entity representation and introduce additional supervision. |
| Outcome: | The proposed framework improves the F1 score on two benchmark datasets by 5.8% . it improves by 6.7% on WebQSP, better than state-of-the-art methods . |
GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction (P19-1)
Copied to clipboard
| Challenge: | GraphRel is an end-to-end relation extraction model that uses graph convolutional networks to learn named entities and relations. |
| Approach: | They propose a graph-based relation extraction model which uses graph convolutional networks to jointly learn named entities and relations. |
| Outcome: | The proposed model outperforms previous models on two public datasets: NYT and WebNLG. |
Joint Entity and Relation Extraction for Legal Documents with Legal Feature Enhancement (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for information extraction are based on pipelining to extract entities from unstructured judgment documents . a large number of judgment documents are released on China Judgments Online . |
| Approach: | They propose a legal triplet extraction system for drug-related criminal judgment documents . they annotate a dataset for Named Entity Recognition and Relation Extraction in Chinese legal domain . |
| Outcome: | The proposed system extracts entities and semantic relations jointly and benefits from the proposed legal lexicon feature and multi-task learning framework. |
Document-level Relationship Extraction by Bidirectional Constraints of Beta Rules (2023.emnlp-main)
Copied to clipboard
| Challenge: | Document-level Relation Extraction (DocRE) aims to extract relations among entity pairs in documents. |
| Approach: | They propose a logic constraint framework that uses bidirectional constraints to model rules by beta contribtion and reconstruct rule consistency loss by bidirectional constraint. |
| Outcome: | The proposed framework outperforms existing models in relation extraction performance and logical consistency. |
Semi-Supervised Tri-Training for Explicit Discourse Argument Expansion (2020.lrec-1)
Copied to clipboard
| Challenge: | a novel application of semi-supervision for shallow discourse parsing is described . we focus on explicit discourse arguments, but we leave the sense selection aside . |
| Approach: | They propose a semi-supervised approach for shallow discourse parsing using sequence tagging. |
| Outcome: | The proposed approach improves performance by 2-10% in the first setting and by comparing the results with training relations. |
Reasoning with Latent Structure Refinement for Document-Level Relation Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for document-level relation extraction capture non-local interactions but are not able to capture rich non-linguistic interactions. |
| Approach: | They propose a document-level relation extraction model that empowers relational reasoning across sentences by automatically inducing the latent document- level graph. |
| Outcome: | The proposed model achieves an F1 score of 59.05 on a large-scale document-level dataset (DocRED), significantly improving over the previous results. |
Event Detection as Graph Parsing (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to event detection focus on using syntactic dependency structures or external knowledge to boost the performance. |
| Approach: | They propose a graph parsing problem that explicitly models multiple event correlations and utilizes rich information conveyed by event type and subtype. |
| Outcome: | The proposed model outperforms existing models on the public ACE2005 dataset by 4.2% on the dataset. |
Non-Parametric Few-Shot Learning for Word Sense Disambiguation (2021.naacl-main)
Copied to clipboard
| Challenge: | Word sense disambiguation (WSD) is a problem in natural language processing . 84% of annotated words have less than 10 examples in the long-tail distribution . |
| Approach: | They propose a non-parametric few-shot learning approach to mitigate word sense disambiguation . they use a metric space to compute distances among the senses of a given word . |
| Outcome: | The proposed method achieves a 75.1 F1 score on the unified evaluation benchmark. |
MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) generate plausible but factually incorrect outputs, posing serious risks to patient safety and clinical decision-making. |
| Approach: | They propose a benchmark for medical hallucination detection using 10,000 question-answer pairs derived from PubMedQA. |
| Outcome: | The proposed model achieves an F1 score as low as 0.625 for detecting 'hard' category hallucinations. |
A Unified Span-Based Approach for Opinion Mining with Syntactic Constituents (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for fine-grained opinion mining (OM) are based on span-based annotations, but they are not effective. |
| Approach: | They propose a unified span-based approach for the end-to-end OM setting using syntactic constituents and multi-task learning to integrate them into the proposed model. |
| Outcome: | The proposed approach achieves significant improvements over previous work on the MPQA 2.0 dataset and reduces the number of wrongly-predicted opinion expressions and roles. |
Teaching Machine Comprehension with Compositional Explanations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in machine reading comprehension rely heavily on large-scale annotated corpora, which are timeconsuming and costly to collect. |
| Approach: | They propose to use semi-structured explanations to “teach” machines reading comprehension using a small number of semi-structural explanations that explicitly inform machines why answer spans are correct. |
| Outcome: | The proposed method achieves 70.14% F1 score with supervision from 26 explanations on the SQuAD dataset, comparable to plain supervised learning using 1,100 labeled instances yielding a 12x speed up. |
TaCube: Pre-computing Data Cubes for Answering Numerical-Reasoning Questions over Tabular Data (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing auto-regressive pre-trained language models are challenged by recent emerging numerical reasoning datasets due to the error-prone implicit calculation. |
| Approach: | They propose a pre-computation tool to pre-compute aggregation/arithmetic results for the table in advance, so they are handy and readily available for PLMs to answer numerical reasoning questions. |
| Outcome: | The proposed model improves on TAT-QA and T5 and BART-large on multiple benchmarks. |
Detecting Denial-of-Service Attacks from Social Media Text: Applying NLP to Computer Security (N18-1)
Copied to clipboard
| Challenge: | Distributed Denial of Service (DDoS) attacks are becoming more frequent and more severe in their impact. |
| Approach: | They propose a feed-forward neural network and a partially labeled LDA model that use social media as an indirect measure of network service status. |
| Outcome: | The proposed model outperforms previous work by significant margins and provides the first fine-grained analysis of how the public reacts to ongoing network attacks. |
Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers? (2024.emnlp-main)
Copied to clipboard
| Challenge: | State-of-the-art Large Language Models (LLMs) are accredited with a number of different capabilities, including reading comprehension, mathematical and reasoning skills, and possessing scientific knowledge. |
| Approach: | They propose a benchmark to generate seemingly plausible multi-hop reasoning chains that ultimately lead to incorrect answers. |
| Outcome: | The proposed model circumvents the reasoning requirement but in subtle ways . it shows that it is more difficult to generate plausible alternatives . |
Is Table Retrieval a Solved Problem? Exploring Join-Aware Multi-Table Retrieval (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for retrieving relevant tables are not sufficient as many questions require retrieving multiple tables and joining them through a join plan that cannot be discerned from the user query itself. |
| Approach: | They propose a method that uncovers useful join relations during table retrieval. |
| Outcome: | The proposed method outperforms the state-of-the-art methods for table retrieval by up to 9.3% in F1 score and for end-to-end QA by up 5.4% in accuracy. |
Adaptive Threshold Selective Self-Attention for Chinese NER (2022.coling-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a computationally difficult task in Chinese since there is no natural delimiter between words in sentences. |
| Approach: | They propose a data-driven Adaptive Threshold Selective Self-Attention mechanism to select the most relevant characters to enhance Transformer architecture for Chinese named entity recognition. |
| Outcome: | Experiments on four benchmark Chinese NER datasets show the proposed mechanism improves performance. |
BIOPSY - Biomarkers In Oncology: Pipeline for Structured Yielding (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Biomarkers are crucial indicators for early cancer detection and prognosis, but extracting biomarkers from clinical texts remains a challenge. |
| Approach: | They propose a pipeline that integrates a domain-adapted biomarker entity recognition model and a relation extraction model to link biomarkers to their respective mutations. |
| Outcome: | The proposed pipeline achieves an F1 score of 0.86 for oncology and 0.87 for neuroscience domains on 5,000 clinical texts. |
InfoSync: Information Synchronization across Multilingual Semi-structured Tables (2023.findings-acl)
Copied to clipboard
| Challenge: | Information Synchronization of semi-structured data across languages is challenging . culture differences, topic preferences, and editing inconsistency lead to information mismatches . |
| Approach: | They propose a method for tabular synchronization that uses information from Wikipedia tables in one language with tables in another language. |
| Outcome: | The proposed method achieves an acceptance rate of 77.28% on Wikipedia . english articles across the web are more timely updated than other languages . |
Capturing Conversational Interaction for Question Answering via Global History Reasoning (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have studied history-dependent reasoning for question answering . utilizing global conversation history for enhancement is gaining interest . |
| Approach: | They propose to establish long-distance dependency among global utterances in multi-turn conversation. |
| Outcome: | The proposed method improves on QuAC by 1%, yielding the F1 score of 73.7%. |
Template-Based Named Entity Recognition Using BART (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fewshot NER do not make full use of knowledge transfer in NER model parameters. |
| Approach: | They propose a template-based method for NER that treats NER as a language model ranking problem in a sequence-to-sequence framework. |
| Outcome: | The proposed method achieves 92.55% F1 score on the CoNLL03 task and significantly better than fine-tuning BERT 10.88%, 15.34%, and 11.73% F1 scores on the MIT Movie, the ATIS, and the MATLAB task. |
Robust Lexical Features for Improved Neural Network Named-Entity Recognition (C18-1)
Copied to clipboard
| Challenge: | Named-Entity Recognition (NER) uses word embeddings to extend, rather than replace, hand-crafted features. |
| Approach: | They propose to embed words and entity types into a low-dimensional vector space and compute a feature vector representing each word offline. |
| Outcome: | The proposed representations outperform existing models and achieve state-of-the-art performance. |
Unintended Bias Detection and Mitigation in Misogynous Memes (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing models that detect misogyny are not able to detect unintended biases in memes, perpetuating harmful stereotypes and reinforcing negative attitudes. |
| Approach: | They propose to measure and mitigate unintentional bias in misogynous memes detection models by using a contextualized scene graph-based multimodal network (CTXSGMNet) they also evaluate their generalizability by evaluating their performance on a few benchmark meme datasets. |
| Outcome: | The proposed model achieves state-of-the-art performance on the SemEval-2022 Task 5 (MAMI task) dataset, showcasing its promising performance in terms of Equity of Odds and F1 score. |
SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for relation extraction only implicitly learn to model relevant contexts and entity types while being trained for RE. |
| Approach: | They propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for RE. |
| Outcome: | The proposed method outperforms the runner-up method on three benchmarks by 5.04% . textual contexts and entity types are the major information sources that lead to the success of previous approaches. |
Bottom-Up Constituency Parsing and Nested Named Entity Recognition with Pointer Networks (2022.acl-long)
Copied to clipboard
| Challenge: | Constituency parsing and nested named entity recognition (NER) are similar tasks since they aim to predict a collection of nesting and non-crossing spans. |
| Approach: | They propose a model that uses a pointer network to predict a constituency tree's boundary . constituency parsing is an important task in natural language processing . |
| Outcome: | The proposed model achieves state-of-the-art performance on PTB among all BERT-based models and competitive performance on CTB7 in constituency parsing. |
TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in generative language models have enabled machines to generate realistic texts. |
| Approach: | They propose a benchmark environment to test the 'Turing Test' problem for neural text generation methods. |
| Outcome: | The proposed benchmark environment is based on 200K human- or machine-generated samples across 20 labels Human, GPT-1, GTP-2_small, GTT-2_medium, GPG-2_large, GGT-2_PyTorch, GGP-3, GROVER_base, griover_large and GRover_mega. |
WikiTalkEdit: A Dataset for modeling Editors’ behaviors on Wikipedia (2021.naacl-main)
Copied to clipboard
| Challenge: | Using the WikiTalkEdit dataset, we show how positive emotion and the use of first-person pronouns predict a positive emotional change in a Wikipedia contributor. |
| Approach: | They introduce and analyze WikiTalkEdit, a dataset of conversations and edit histories from Wikipedia, for research in online cooperation and conversation modeling. |
| Outcome: | The proposed dataset supports the classic understanding of style matching, where positive emotion and the use of first-person pronouns predict a positive emotional change in a Wikipedia contributor. |
Dynamic Emotion and Personality Profiling for Multimodal Deception Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for deception detection lack sample-level dynamic annotations for emotions and personality. |
| Approach: | They propose a multi-model multi-prompt annotation scheme and a strict label quality evaluation standard for deception, emotion, and personality annotations. |
| Outcome: | The proposed framework outperforms state-of-the-art models on the MDPE and DDEP datasets. |
Glocal: Incorporating Global Information in Local Convolution for Keyphrase Extraction (N19-1)
Copied to clipboard
| Challenge: | Graph Convolutional Networks (GCNs) model nodes’ local pairwise importance but lack the capability to model global relative importance in tasks where global ranking is a key component for the task. |
| Approach: | They propose to incorporate global relative importance information into the GCN family of models by using scaled node weights. |
| Outcome: | The proposed method improves keyphrase extraction by 2% and improves the baseline by 5%. |
GenerativeRE: Incorporating a Novel Copy Mechanism and Pretrained Model for Joint Entity and Relation Extraction (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for extracting relation triplets suffer from incompletion and disorder problems when they extract multi-token entities from input sentences. |
| Approach: | They propose a special entity labelling method that fine-tunes the pre-trained model and learns the special entity labels simultaneously. |
| Outcome: | The proposed model achieves 4.6% and 0.9% improvement over current methods in the NYT24 and NYT29 benchmark datasets. |
Uncovering Implicit Inferences for Improved Relational Argument Mining (2023.eacl-main)
Copied to clipboard
| Challenge: | Argument mining attempts to extract arguments and their structure from unstructured texts. |
| Approach: | They propose a generative neuro-symbolic approach to finding inference chains that connect argument pairs by using the Commonsense Transformer. |
| Outcome: | The proposed approach outperforms the state-of-the-art by 2-5% in F1 score on three datasets. |
BanglaRQA: A Benchmark Dataset for Under-resourced Bangla Language Reading Comprehension-based Question Answering with Diverse Question-Answer Types (2022.findings-emnlp)
Copied to clipboard
Syed Mohammed Sartaj Ekram, Adham Arik Rahman, Md. Sajid Altaf, Mohammed Saidul Islam, Mehrab Mustafy Rahman, Md Mezbaur Rahman, Md Azam Hossain, Abu Raihan Mostofa Kamal
| Challenge: | a lack of diverse and comprehensive question-answering datasets exists in under-resourced languages like Bangla. |
| Approach: | They propose a reading comprehension-based Bangla question-answering dataset . the dataset includes answerable and unanswerable questions covering four categories of questions . |
| Outcome: | The proposed dataset shows that it performs well as a training resource in high-resource languages. |
DocQueryNet: Value Retrieval with Arbitrary Queries for Form-like Documents (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods that only address a fixed set of fields are difficult to use for different form types. |
| Approach: | They propose a value retrieval method with arbitrary queries for form-like documents . they propose 'docQueryNet' to predict target value based on understanding of layout and semantics of a form . |
| Outcome: | The proposed method outperforms existing methods on value retrieval . it improves document understanding on large-scale model pre-training by 17% . |
A Language Model-based Generative Classifier for Sentence-level Discourse Parsing (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to consider textual coherence are limited in labeled data. |
| Approach: | They propose a language model-based generative classifier that uses labels as input and embeds labels into their representations. |
| Outcome: | The proposed classifier achieves state-of-the-art in discourse segmentation and relation F1 scores with gold boundaries and automatically segmented boundaries. |
Neural Combinatory Constituency Parsing (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to constituency parsing are based on symbolic engineering, but they are simplified by their adaptive distributed representation. |
| Approach: | They propose two fast combinatory models for constituency parsing: binary and multibranching. |
| Outcome: | The proposed models achieve an F1 score of 92.54 on Penn Treebank, speeding at 1327.2 sents/sec. |
Cross-lingual Decompositional Semantic Parsing (D18-1)
Copied to clipboard
| Challenge: | Renewed interest in semantic analysis has led to a surge of proposed new frameworks . many of these efforts are limited to the analysis of English, but with a number of exceptions e.g., recent efforts in Minimal Recursion Semantics (MRS) and multilingual FrameNet annotation and parsing. |
| Approach: | They propose a cross-lingual decompositional semantic analysis task based on a target language . they propose 'end-to-end' model with an annotating mechanism that supports intra-sentential coreference . |
| Outcome: | The proposed model outperforms baselines by at least 1.75 F1 score on an evaluation dataset. |
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices (2020.acl-main)
Copied to clipboard
| Challenge: | Empirical studies show that MobileBERT is 4.3x smaller and 5.5x faster than BERT_BASE . BERT is one of the largest models ever in NLP, but suffers from heavy model size and high latency . |
| Approach: | They propose a tool to compress and accelerate the popular BERT model by task-agnostic application. |
| Outcome: | The proposed model is 4.3x smaller and 5.5x faster than BERT_BASE . it achieves competitive results on well-known benchmarks . |
Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning has emerged as a promising approach to resolve anaphora, but there are challenges in applying it to scientific protocols. |
| Approach: | They propose a method which combines predictions of hundreds of in-context experts and combines them to yield a 30% increase in F1 over a competitive prompt retrieval baseline. |
| Outcome: | The proposed method yields 30% increase in F1 score over a competitive prompt retrieval baseline. |
Suicidal Risk Detection for Military Personnel (2020.emnlp-main)
Copied to clipboard
| Challenge: | a dataset of 2,791 posts with 13,955 expert annotations of suicidal risk levels is available for research . Suicide is one of the major causes of death in the military. |
| Approach: | They analyze posts related to military service in the Republic of Korea and annotate them with military experts and mental health experts. |
| Outcome: | The proposed method predicts the level of suicide risk, reaching .88 F1 for classifying the risks. |
GENEVA: Benchmarking Generalizability for Event Argument Extraction with Hundreds of Event Types and Argument Roles (2023.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarking datasets for Event Argument Extraction (EAE) cover less than 40 event types and 25 entity-centric argument roles. |
| Approach: | They propose to use a large and diverse EAE ontology to create a semantic role labeling dataset for EAE that incorporates 115 events and 220 argument roles. |
| Outcome: | The proposed ontology concludes with 115 events and 220 argument roles, with a significant portion of roles not being entities. |
NerKor+Cars-OntoNotes++ (2022.lrec-1)
Copied to clipboard
| Challenge: | In this paper, we present an upgraded version of the Hungarian NYTK-NerKor named entity corpus . it contains twice as many annotated spans and 7 times as many distinct entity types as the original version. |
| Approach: | They present an upgraded version of the Hungarian NYTK-NerKor named entity corpus with an extended OntoNotes 5 annotation scheme. |
| Outcome: | The enhanced version of the corpus contains twice as many annotated spans and 7 times more distinct entity types than the original version. |
SLANG-GraphRAG: Multi-Layered Retrieval with Domain-Specific Knowledge for Low Resource Social Media Conversations (2026.findings-eacl)
Copied to clipboard
| Challenge: | Standard NLP benchmarks often miss subtle, culturally-specific cues in social media . incorporating structured cultural knowledge into the retrieval process improves accuracy by up to 31% . |
| Approach: | They propose a retrieval-augmented framework that integrates a culture-specific slang knowledge graph into large language models via one-shot prompting. |
| Outcome: | The proposed framework outperforms traditional and unstructured retrieval methods in slang-based models by 31% and 28%. |
Bridging the Gap Between BabelNet and HowNet: Unsupervised Sense Alignment and Sememe Prediction (2023.eacl-main)
Copied to clipboard
| Challenge: | Sememes are the minimum semantic units of natural languages, but their use is limited by a lack of available sememe knowledge bases. |
| Approach: | They propose to use sense alignment to connect BabelNet with HowNet by relaxing constraints until a complete alignment is achieved. |
| Outcome: | The proposed method improves on previous supervised methods by 12% . it is based on interpretable propagation of sememe information between lexical resources . |
Rethinking Negative Instances for Generative Named Entity Recognition (2024.findings-acl)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) models are constrained by a pre-defined label set and require extensive human annotations, which limits their flexibility and adaptability to unseen tasks. |
| Approach: | They propose a Generative NER system that shows improved zero-shot performance across unseen entity domains by introducing contextual information and delineating label boundaries. |
| Outcome: | The proposed model outperforms state-of-the-art methods in zero-shot evaluation. |
Uncertain Local-to-Global Networks for Document-Level Event Factuality Identification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on identifying event factuality at sentence level, which leads to conflicts between different mentions of the same event. |
| Approach: | They propose a document-level event factuality identification model that uses local uncertainty and global structure to model event factuality. |
| Outcome: | The proposed method outperforms existing models on two widely used datasets. |
Self-contradictory reasoning evaluation and detection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown impressive reasoning ability, but many downstream reasoning tasks focus on performance-wise evaluation. |
| Approach: | They define and assess the Self-Contra rate across three datasets and delve into finer-grained categories of Self-contra reasoning. |
| Outcome: | The proposed model can detect self-contra reasoning with a 52.2% F1 score, much lower than for humans. |
COF: Adaptive Chain of Feedback for Comparative Opinion Quintuple Extraction (2025.coling-main)
Copied to clipboard
| Challenge: | Comparative Opinion Quintuple Extraction (COQE) aims to extract all comparative sentiment quintuples from product review text. |
| Approach: | They propose a model-unaware adaptive chain-of-feedback method to extract quintuples from product review text. |
| Outcome: | The proposed method improves performance on three benchmarks. |
Urdu Word Segmentation using Conditional Random Fields (CRFs) (C18-1)
Copied to clipboard
| Challenge: | Urdu is amongst languages for which word segmentation is a complex task as it exhibits space omission and space insertion issues. |
| Approach: | They propose a word segmentation system for Urdu which uses a Conditional Random Field sequence modeler with orthographic, linguistic and morphological features. |
| Outcome: | The proposed system achieves an F1 score for word boundary identification and 0.85 for sub-word boundary identification tasks. |
Entity Relation Extraction as Dependency Parsing in Visually Rich Documents (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on key information extraction from visually rich documents focus on labeling the text within bounding boxes, while relations between words are unexplored. |
| Approach: | They propose to use a dependency parsing model to extract semantic entities from visually rich documents by combining entity labeling and relation extraction tasks. |
| Outcome: | The proposed model achieves 65.96% F1 score on the FUNSD dataset. |
SWEET - Weakly Supervised Person Name Extraction for Fighting Human Trafficking (2023.findings-emnlp)
Copied to clipboard
| Challenge: | SWEET is a weak supervision pipeline for extracting person names from noisy escort ads . it does not require any human annotators and labeling, which is incredibly important . |
| Approach: | They propose a weak supervision pipeline SWEET: Supervise Weakly for Entity Extraction to fight Trafficking for extracting person names from noisy escort ads. |
| Outcome: | The proposed weak supervision pipeline outperforms the previous method by 9% on domain data and generalizes to common benchmark datasets. |
Breakthrough from Nuance and Inconsistency: Enhancing Multimodal Sarcasm Detection with Context-Aware Self-Attention Fusion and Word Weight Calculation. (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for sarcasm detection rely on feature concatenation to fuse different modalities or model inconsistencies among modalités. |
| Approach: | They propose to use Context-Aware Self-Attention Fusion to integrate local and momentary multimodal information into specific words to illustrate the inconsistencies between connotation and denotation. |
| Outcome: | The proposed method achieves an accuracy of 76.9 and an F1 score of 76.1 on the MUStARD dataset, surpassing the current state-of-the-art IWAN model by 1.7 and 1.6 respectively. |
Improving Numerical Reasoning Skills in the Modular Approach for Complex Question Answering on Text (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Module Networks (NMNs) is an end-to-end differentiable model in the programmer-interpreter paradigm. |
| Approach: | They propose to make the interpreter question-aware and capture the relationship between entities and numbers in both questions and paragraphs. |
| Outcome: | The proposed models outperform the original models on the DROP dataset and are interpertable by nature. |
SpecNFS: A Challenge Dataset Towards Extracting Formal Models from Natural Language Specifications (2022.lrec-1)
Copied to clipboard
Sayontan Ghosh, Amanpreet Singh, Alex Merenstein, Wei Su, Scott A. Smolka, Erez Zadok, Niranjan Balasubramanian
| Challenge: | Existing methods for building formal semantic representations of specification texts are laborious and error-prone. |
| Approach: | They propose to use SpecIR to model sentences appearing in NFS specification documents as IF-THEN statements and introduce a representation language to parse them. |
| Outcome: | The proposed models achieve an F1 score of only 60.5 and 33.3 when using a state-of-the-art language model. |
MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages (2023.acl-long)
Copied to clipboard
Jack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie, Kay Rottmann, Ana Sanchez, Aaron Nash, Liam Urbach, Vishesh Kakarala, Richa Singh, Swetha Ranganath, Laurie Crist, Misha Britan, Wouter Leeuwis, Gokhan Tur, Prem Natarajan
| Challenge: | We present the MASSIVE dataset–Multilingual Amazon Slu resource package (SLURP) for Slot-filling, Intent classification, and Virtual assistant evaluation. |
| Approach: | They present a 1M-example dataset of Amazon Slu utterances . they localize the dataset into 50 typologically diverse languages . |
| Outcome: | The proposed model includes exact match accuracy, intent classification accuracy, and slot-filling F1 score. |
FQuAD2.0: French Question Answering and Learning When You Don’t Know (2022.lrec-1)
Copied to clipboard
| Challenge: | Question Answering, including Reading Comprehension, has seen significant scientific breakthroughs over the past few years . but most of these breakthroughs are centered on the English language . |
| Approach: | They propose a dataset to train Question Answering models in the French language . they extend the dataset to 17,000+ unanswerable questions annotated adversarially . |
| Outcome: | The proposed dataset makes it possible to train French Question Answering models with the ability to distinguish unanswerable questions from answerable ones. |
Explanations for CommonsenseQA: New Dataset and Models (2021.acl-long)
Copied to clipboard
Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal, Parag Singla, Dinesh Garg
| Challenge: | a dataset called CommonsenseQA (CQA) was recently released to advance the research on common-sense question answering (QA) |
| Approach: | They propose to retrieve and generate explanations for a given question, correct answer choice, incorrect answer choices tuple from a dataset called CommonsenseQA. |
| Outcome: | The proposed model beats baseline model by 100% in F1 score and similarity score of 61.9 . |
Low Resource Sequence Tagging using Sentence Reconstruction (2020.acl-main)
Copied to clipboard
| Challenge: | This work revisits the task of training sequence tagging models with limited resources using transfer learning. |
| Approach: | They propose a new loss that relies on sentence reconstruction from normalized embeddings. |
| Outcome: | The proposed method achieves 0.6 F1 score on Dutch datasets using only one sample. |
OpenEL: An Annotated Corpus for Entity Linking and Discourse in Open Domain Dialogue (2022.lrec-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER), named entity linking and discourse modeling are crucial aspects of natural language understanding for open domain dialogue systems. |
| Approach: | They present an annotated multi-domain corpus for linking entities in open-domain dialogue . they use dialogue context and anaphora resolution to assess the effectiveness of the task . |
| Outcome: | The OpenEL corpus is an annotated multi-domain corpus for linking entities in open-domain dialogue . the system Flair + BLINK has the best performance with a 0.65 F1 score . |
iACOS: Advancing Implicit Sentiment Extraction with Informative and Adaptive Negative Examples (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for extracting aspects and opinions from text are incomplete. |
| Approach: | They propose a method for extracting Implicit Aspects with Categories and Opinions with Sentiments using implicit tokens. |
| Outcome: | The proposed method outperforms baseline methods on two public benchmark datasets. |
The Causal News Corpus: Annotating Causal Relations in Event Sentences from News (2022.lrec-1)
Copied to clipboard
Fiona Anting Tan, Ali Hürriyetoğlu, Tommaso Caselli, Nelleke Oostdijk, Tadashi Nomoto, Hansi Hettiarachchi, Iqra Ameer, Onur Uca, Farhana Ferdousi Liza, Tiancheng Hu
| Challenge: | Existing annotation guidelines for event causality focus on only explicit relations or clauses. |
| Approach: | They propose an annotation schema for event causality that addresses these concerns . they annotated 3,559 event sentences from protest event news with labels on whether it contains causal relations or not. |
| Outcome: | The proposed annotation schema for event causality addresses these concerns . it performs well with 81.20% F1 score on test set and 83.46% in 5-folds cross-validation . |
Multi-Task Learning for Conversational Question Answering over a Large-Scale Knowledge Base (D19-1)
Copied to clipboard
| Challenge: | Recent approaches to handle large knowledge base decompose tasks into subtasks and solve them sequentially. |
| Approach: | They propose a multi-task learning framework that resolves coreference in conversations . they propose enabling shared supervisions and type-aware entity detection model . |
| Outcome: | The proposed framework improves overall F1 score from 67% to 79% on a large-scale conversational question answering dataset. |
HacRED: A Large-Scale Relation Extraction Dataset Toward Hard Cases in Practical Applications (2021.findings-acl)
Copied to clipboard
Qiao Cheng, Juntao Liu, Xiaoye Qu, Jin Zhao, Jiaqing Liang, Zhefeng Wang, Baoxing Huai, Nicholas Jing Yuan, Yanghua Xiao
| Challenge: | Relation extraction (RE) is an essential topic in natural language processing and has attracted extensive attention. |
| Approach: | They propose a case-oriented construction framework to build a hard case relation extraction dataset with 65,225 relational facts annotated from 9,231 documents. |
| Outcome: | The proposed model achieves a high 96% F1 score on data quality and is far lower than humans. |
BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novels (D19-1)
Copied to clipboard
| Challenge: | Using BiPaR, we build monolingual, multilingual and cross-lingual MRC on novels. |
| Approach: | They propose a bilingual parallel novel-style machine reading comprehension dataset BiPaR . they collect 3,667 bilingual parallel paragraphs from Chinese and English novels . |
| Outcome: | The proposed dataset supports multilingual and cross-lingual reading comprehension. |
Read between the lines - Functionality Extraction From READMEs (2024.findings-naacl)
Copied to clipboard
| Challenge: | While text summarization is a well-known NLP task, functionality extraction from Git README files is based on a novel and useful variant of it. |
| Approach: | They propose a novel task called functionality extraction from Git README files that detects all the functionalities supported by the corresponding application software. |
| Outcome: | The proposed model outperforms baseline models that can be designed using popular black-box or white-box large language models (LLMs) such as ChatGPT and Bard. |
Understanding the Cooking Process with English Recipe Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to translate recipes into a flow graph have performance problems . authors propose a framework to construct a graph from recipe text . |
| Approach: | They propose a framework that can be used to translate recipes into a flow graph representation. |
| Outcome: | The proposed framework can predict the edge label and achieve the overall F1 score of 92.2 on the English recipe flow graph corpus. |
ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to relation extraction require labeled data, but labeling is difficult . Existing models cannot recognize rare instances that are never covered by training data . |
| Approach: | They propose a multi-task learning model that directly predicts unseen relations without hand-crafted attribute labeling and multiple pairwise classifications. |
| Outcome: | The proposed model outperforms existing methods by 13.54% on two well-known datasets. |
Detecting Scenes in Fiction: A new Segmentation Task (2021.eacl-main)
Copied to clipboard
Albin Zehe, Leonard Konle, Lea Katharina Dümpelmann, Evelyn Gius, Andreas Hotho, Fotis Jannidis, Lucas Kaufmann, Markus Krug, Frank Puppe, Nils Reiter, Annekea Schreiber, Nathalie Wiedmer
| Challenge: | Text segmentation is a long standing issue in the area of natural language processing . even modern methods struggle with processing text longer than a couple of sentences or paragraphs . |
| Approach: | They introduce the task of scene segmentation on narrative texts and provide an annotated corpus . they discuss linguistic and narrative properties of the task and provide baseline experiments . |
| Outcome: | The proposed task is very challenging and the results are impressive. |
Explicit Alignment Objectives for Multilingual Bidirectional Encoders (2021.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained cross-lingual encoders have proven impressively effective at enabling transfer-learning of NLP systems from high-resource languages to low-resourced languages. |
| Approach: | They propose a method to align multilingual encoders using two explicit alignment objectives that align the multilingual representations at different granularities. |
| Outcome: | The proposed method achieves gains of up to 1.1 average F1 score on sequence tagging and 27.3 average accuracy on retrieval over the XLM-R-large model. |
ECPE-2D: Emotion-Cause Pair Extraction based on Joint Two-Dimensional Representation, Interaction and Prediction (2020.acl-main)
Copied to clipboard
| Challenge: | a new task, called emotion-cause pair extraction, has emerged in text emotion analysis . a 2D representation scheme is proposed to represent the emotion-case pairs . |
| Approach: | They propose a 2D approach to represent emotion-cause pairs by a 3D representation scheme. |
| Outcome: | The proposed approach improves the state-of-the-art on the emotion cause corpus . the proposed approach is based on a two-step framework with flaws . |
DocEE: A Large-Scale and Fine-grained Benchmark for Document-level Event Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets focus on sentence-level event extraction, but document-level EE is limited due to the lack of large-scale and practical training and evaluation datasets. |
| Approach: | They propose a document-level event extraction dataset with 27,000+ events and 180,000+ arguments. |
| Outcome: | The proposed dataset includes 27,000+ events, 180,000+ arguments and large-scale manual annotations, fine-grained argument types and application-oriented settings. |
Syntax-Aware Opinion Role Labeling with Dependency Graph Convolutional Networks (2020.acl-main)
Copied to clipboard
| Challenge: | Opinion role labeling (ORL) is a fine-grained opinion analysis task . due to the scarcity of labeled data, ORL remains challenging for data-driven methods due to its complexity and complexity. |
| Approach: | They propose to integrate syntactic knowledge into ORL models by comparing and integrating different representations and using dependency graph convolutional networks to encode parser information at different processing levels. |
| Outcome: | The proposed model achieves 4.34 higher F1 score than the current state-of-the-art. |
An Unsupervised Method for Learning Representations of Multi-word Expressions for Semantic Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for learning multi-word expressions have language sparsity and are not supervised. |
| Approach: | They propose an unsupervised approach to learning a compositional representation function for multi-word expressions . they use a Tratz dataset to train the composition function on the word-semantic relation . |
| Outcome: | The proposed method outperforms the previous state-of-the-art method on the Tratz dataset with an F1 score of 50.4%. |
Learning multiview embeddings for assessing dementia (D18-1)
Copied to clipboard
| Challenge: | In 2017, 5.7 million Americans were living with Alzheimer's disease (AD), and the disease accounted for $11.4 billion in healthcare costs in the United States. |
| Approach: | They leverage the multiview nature of a small AD dataset to learn an embedding that captures different modes of cognitive impairment. |
| Outcome: | The proposed embeddings achieve an F1 score of 0.82 and a mean absolute error of 3.42 in the classification task and predicting clinical scores. |
Source Critical Reinforcement Learning for Transferring Spoken Language Understanding to a New Language (C18-1)
Copied to clipboard
| Challenge: | a study aims to develop a language transferring system to avoid the trouble of acquiring and labeling a new big SLU corpus . general-purpose translators cannot handle the lot of semantic labels, not to mention cultural differences . a RL-based language transfer method can be used to adapt the adapted translator to a target language . |
| Approach: | They propose to use reinforcement learning to adapt a spoken language understanding model to a target language. |
| Outcome: | The proposed language transferring method improves domain classification accuracy by 22% compared with naive translation . the proposed language transfer method can be used on Chinese to English translators with more proper slot tags . |
Attention for Implicit Discourse Relation Recognition (L18-1)
Copied to clipboard
| Challenge: | Existing approaches to implicit discourse relation recognition reach F1 scores of 9.95% to 37.67% . a neural network exploits the strong correlation between pairs of words that implicitly signal a discourse relation. |
| Approach: | They propose a neural network which exploits strong correlation between pairs of words . they use an encoder-decoder model with attention to detect a latent discourse relation . |
| Outcome: | The proposed model outperforms state-of-the-art models on fine-grained classification and fine-granular classification while computing parameters without pooling and fully connected layers. |
A general framework for information extraction using dynamic span graphs (N19-1)
Copied to clipboard
| Challenge: | Existing frameworks for information extraction use a pipeline approach to identify entities and then use the detected entity spans for relation extraction and coreference resolution. |
| Approach: | They propose a framework for several information extraction tasks that share span representations using dynamically constructed span graphs. |
| Outcome: | The proposed framework significantly outperforms state-of-the-art on multiple information extraction tasks across multiple datasets reflecting different domains. |
Unsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for unsupervised Chinese word segmentation exploit shallow semantic information, which can miss important context. |
| Approach: | They propose to take advantage of deep contextual semantic information with a self-training manner to transform it into explicit word segmentation ability. |
| Outcome: | The proposed approach achieves state-of-the-art F1 score on two CWS benchmark datasets. |
Federated Learning for Spoken Language Understanding (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to improve robustness of models focus on a single dataset . but, there are few studies on how to combine merits of different datasets . |
| Approach: | They propose a federated learning framework that could unify datasets and tasks . they propose MV-Encoder as backbone of the framework to provide multi-granularity text representations . |
| Outcome: | The proposed framework improves on two SLU benchmark datasets and federated learning settings. |
NormMark: A Weakly Supervised Markov Model for Socio-cultural Norm Discovery (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for norm recognition focus only on surface-level features of dialogues and do not take into account the interactions within a conversation. |
| Approach: | They propose a probabilistic generative Markov model to carry latent features throughout a dialogue and trainable on weakly annotated data using the variational technique. |
| Outcome: | The proposed model outperforms current state-of-the-art methods on a weakly annotated dataset, outperforming existing methods, including GPT3. |
LogicST: A Logical Self-Training Framework for Document-Level Relation Extraction with Incomplete Annotations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Document-level relation extraction (DocRE) is difficult due to the vast number of entity pairs. |
| Approach: | They propose a neural-logic self-training framework that iteratively resolves conflicts and constructs the minimal diagnostic set for updating models. |
| Outcome: | The proposed framework outperforms existing methods on the document-level relation extraction (docRE) benchmark. |
GLaRA: Graph-based Labeling Rule Augmentation for Weakly Supervised Named Entity Recognition (2021.eacl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) models often need to be trained with many manual labels to perform well. |
| Approach: | They propose to train named entity recognition systems using heuristic labeling rules . they create a graph with nodes representing candidate rules extracted from unlabeled data . |
| Outcome: | The proposed method achieves an average improvement of +20% over the baseline on three NER datasets. |
On the Role of Key Phrases in Argument Mining (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to argument mining often overlook crucial conceptual links between ACs and ARs. |
| Approach: | They propose a framework that extracts key phrases from AM benchmarks using an open-source Large Language Model. |
| Outcome: | The proposed framework surpasses baselines on three structurally distinct AM benchmarks by up to 9.5% F1 score, demonstrating its strong potential. |
Evaluating Multimodal Large Language Models on Video Captioning via Monte Carlo Tree Search (2025.acl-long)
Copied to clipboard
Linhao Yu, Xingguang Ji, Yahui Liu, Fanheng Kong, Chenxi Sun, Jingyuan Zhang, Hongzhi Zhang, V. W., Fuzheng Zhang, Deyi Xiong
| Challenge: | Existing benchmarks and evaluation protocols suffer from inadequate or homogeneous creation of key points, exorbitant cost of data creation, and limited evaluation scopes. |
| Approach: | They propose an automatic framework which leverages Monte Carlo Tree Search to construct numerous and diverse descriptive sentences that thoroughly represent video content in an iterative way. |
| Outcome: | The proposed framework improves MCTS-VCB and DREAM-1K on video captioning tasks by 25.0% and 16.3% respectively. |
Evaluating Pretrained Transformer-based Models on the Task of Fine-Grained Named Entity Recognition (2020.coling-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a fundamental task in Natural Language Processing (NLP). |
| Approach: | They compare three transformer-based names to two non-transformer-based ones . they find transformer-derived models incrementally outperform non-tranformer models . |
| Outcome: | The proposed models outperform the studied models in most domains with respect to the F1 score. |
Named Entity Recognition through Deep Representation Learning and Weak Supervision (2021.findings-acl)
Copied to clipboard
| Challenge: | Weakly supervised named entity recognition (NER) uses noisy labels to estimate the true labels of a dataset. |
| Approach: | They propose a model to learn optimal assignments of latent NER tags using observed tokens and weak labels provided by labeling functions. |
| Outcome: | The proposed model improves the quality of weak labels on four public datasets. |
Leveraging Training Dynamics and Self-Training for Text Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semi-supervised learning (SSL) is a promising technique for improving deep learning models when training data is scarce. |
| Approach: | They propose a semi-supervised learning approach that leverages training dynamics of unlabeled data. |
| Outcome: | The proposed method achieves an average increase in F1 score of 3.5% over baselines in low resource settings. |
Can LLMs Recognize Toxicity? A Structured Investigation Framework and Toxicity Metric (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing toxicity metrics rely on encoder models trained on specific toxicity datasets, which are susceptible to out-of-distribution (OOD) problems and depend on the dataset’s definition of toxicity. |
| Approach: | They propose a robust metric grounded on LLMs to flexibly measure toxicity according to the given definition by analysing toxicity factors and intrinsic toxic attributes. |
| Outcome: | The proposed metric improves on conventional metrics by 12 points in the F1 score and shows that upstream toxicity significantly influences downstream metrics, suggesting that LLMs are unsuitable for toxicity evaluations within unverified factors. |
Literary Event Detection (P19-1)
Copied to clipboard
| Challenge: | a new dataset of literary events is presented to examine the nature of narratives . literature presents a number of challenges for existing systems, including complex narration . |
| Approach: | They propose a dataset of literary events that are depicted as taking place within the imagined space of a novel. |
| Outcome: | The proposed model achieves an F1 score of 73.9 for prestige and popularity . the best performing model achieve a score of 79.9 for prestige compared to the previous model . |
Fine-tuned LLMs Know More, Hallucinate Less with Few-Shot Sequence-to-Sequence Semantic Parsing over Wikidata (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models can answer many questions correctly, but can also hallucinate and give wrong answers. |
| Approach: | They propose a question-answering benchmark for Wikidata that uses SPARQL to ground large language models. |
| Outcome: | The proposed method outperforms the state-of-the-art for QALD-7 by 3.6% in F1 score. |
Advancing Multi-Criteria Chinese Word Segmentation Through Criterion Classification and Denoising (2023.acl-long)
Copied to clipboard
| Challenge: | Recent research on multi-criteria Chinese word segmentation focuses on building complex private structures, adding more handcrafted features, or introducing complex optimization processes. |
| Approach: | They propose a model that fits multiple Chinese word segments using input-hint inputs. |
| Outcome: | The proposed model achieves state-of-the-art (SoTA) performance on multiple datasets simultaneously. |
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks focus on pairwise matching, ignoring robustness . current models exhibit frustrating degradation, with a maximum drop of 23.43 F1 score . |
| Approach: | They propose a benchmark that simulates the evaluation of open information extraction models in the real world . they perform experiments on typical models published in the last decade and a representative large language model . |
| Outcome: | The proposed model is rated robust on a knowledge-invariant clique with different syntactic and expressive forms. |
Unleashing Multi-Hop Reasoning Potential in Large Language Models through Repetition of Misordered Context (2025.findings-naacl)
Copied to clipboard
| Challenge: | Multi-hop reasoning requires multi-step reasoning based on supporting documents within a given context. |
| Approach: | They propose a method that prompts the model by repeatedly presenting the context. |
| Outcome: | The proposed method improves the F1 score by 30%p on multi-hop QA tasks and increases accuracy by 70%p on a synthetic task. |
Structural Scaffolds for Citation Intent Classification in Scientific Publications (N19-1)
Copied to clipboard
| Challenge: | Existing methods for identifying intent of citations are limited by external linguistic resources and hand-engineered features. |
| Approach: | They propose a multitask model to incorporate structural information of scientific papers into citations for effective classification of citation intents. |
| Outcome: | The proposed model achieves a 13.3% increase in F1 score on an existing ACL anthology dataset without external linguistic resources or hand-engineered features as done in existing methods. |
Time-Stamped Language Model: Teaching Language Models to Understand The Flow of Events (2021.naacl-main)
Copied to clipboard
| Challenge: | Using transformer-based language models to track entities is challenging due to dynamic nature of the world described in the text. |
| Approach: | They propose to use transformer-based language models to track entities throughout a procedure . they propose to introduce timestamp encoding to encode event information in LMs . |
| Outcome: | The proposed model improves on the state-of-the-art model with a 3.1% increase in F1 score on the Propara dataset and better results on the location prediction task on the NPN-Cooking dataset. |
Improved Word Sense Disambiguation with Enhanced Sense Representations (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing supervised word sense disambiguation systems do not provide enough information about word senses. |
| Approach: | They propose to incorporate synonyms, example phrases or sentences showing usage of word senses and sense gloss of hypernyms into the sense representations. |
| Outcome: | The proposed system achieves an F1 score of 82.0% on the standard benchmark test dataset of the English all-words WSD task. |
KinyaBERT: a Morphology-aware Kinyarwanda Language Model (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models such as BERT are sub-optimal at handling morphologically rich languages. |
| Approach: | They propose a two-tier BERT architecture that leverages a morphological analyzer and explicitly represents morphology in a low-resource Kinyarwanda language. |
| Outcome: | The proposed model outperforms baseline models on the low-resource morphologically rich Kinyarwanda language by 2% in F1 score and 4.3% in average score of GLUE benchmark. |
Nebula: A discourse aware Minecraft Builder (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has shown that at least some context is needed to understand and carry out conversationally given instructions. |
| Approach: | They propose to incorporate prior discourse and nonlinguistic contexts of a conversation situated in a nonlinguistic environment into an LLM model to improve the "language to action" component of collaborative tasks. |
| Outcome: | The proposed model doubles the baseline on the task of Jayannavar et al. (2020) and can construct shapes and understand location descriptions using a synthetic dataset. |
Robust Frame-Semantic Models with Lexical Unit Trees and Negative Samples (2024.acl-long)
Copied to clipboard
| Challenge: | Using a RoBERTa-based filter, we achieve an F1 score of 0.775, surpassing the previous state-of-the-art solution by +0.012. |
| Approach: | They propose a new prefix tree modification to enable robust support for multi-word lexical units and a RoBERTa-based filter to achieve an F1 score of 0.775. |
| Outcome: | The proposed model achieves an F1 score of 0.775, surpassing the state-of-the-art model by +0.012. |
ReFACT: A Benchmark for Scientific Confabulation Detection with Positional Error Annotations (2026.eacl-long)
Copied to clipboard
Yindong Wang, Martin Preiß, Margarita Bugueño, Jan Vincent Hoffbauer, Abdullatif Ghajar, Tolga Buz, Gerard de Melo
| Challenge: | Evaluating 9 state-of-the-art LLMs reveals two critical limitations: 61% of incorrect span predictions are semantically unrelated to actual errors. |
| Approach: | They propose a benchmark of 1,001 expert-annotated question-answer pairs with span-level error annotations derived from Reddit's r/AskScience. |
| Outcome: | Evaluating 9 state-of-the-art LLMs, we find that comparative judgment is paradoxically harder than independent detection when comparing answers side-by-side. |
Structure-Discourse Hierarchical Graph for Conditional Question Answering on Long Documents (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to conditional question answering on long documents ignore document structure and discourse relations between sentences in document sections. |
| Approach: | They construct a Structure-Discourse Hierarchical Graph and conduct bottom-up information propagation to address this issue. |
| Outcome: | The proposed approach outperforms the existing methods on the conditional question answering on long documents by 3.0 EM score and 2.4 F1 score on answer measuring, and 2.2 EM and 1.9 F1 scores on jointly answer and condition measuring. |
LILA: A Unified Benchmark for Mathematical Reasoning (2022.emnlp-main)
Copied to clipboard
Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, Ashwin Kalyan
| Challenge: | Towards evaluating and improving AI systems in this domain, we propose a mathematical reasoning benchmark based on 23 diversetasks . |
| Approach: | They propose a mathematical reasoning benchmark that includes 23 diverse tasks . they extend the benchmark by collecting task instructions and solutions in the form of Python programs . |
| Outcome: | The proposed model improves on multi-tasking while the best performing model only achieves 60.40%. |
Large Dataset and Language Model Fun-Tuning for Humor Recognition (P19-1)
Copied to clipboard
| Challenge: | Humor recognition datasets contain only English texts and focus on puns. |
| Approach: | They collected a dataset of jokes and funny dialogues in Russian and complemented them carefully with unfunny texts with similar lexical properties. |
| Outcome: | The proposed method is based on the universal language model finetuning and has an F1 score of 0.91 on a test set. |
Syntactically Rich Discriminative Training: An Effective Method for Open Information Extraction (2022.emnlp-main)
Copied to clipboard
| Challenge: | Open information extraction (OIE) is the task of extracting facts from natural language text. |
| Approach: | They propose a method for computing syntactically rich text embeddings using the structure of dependency trees and a discriminative approach to OIE where tokens in the generated fact are classified as "real" and "fake" they propose to reduce repetitive tokens and improve models' ability to generate implicit facts by a factor of 23%. |
| Outcome: | The proposed method reduces repetitive tokens by a factor of 23% on the CaRB, OIE2016, and LSOIE datasets and improves on augmented datasets. |
Enhancing Dialogue-based Relation Extraction by Speaker and Trigger Words Prediction (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for identifying relations from dialogues do not fully consider the particularity of dialogues, making them difficult to understand the semantics between conversational arguments. |
| Approach: | They propose two tasks to enhance the extraction of dialogue-based relations . speaker prediction captures the characteristics of speakerrelated entities . the trigger words prediction provides supportive contexts for relations between arguments . |
| Outcome: | The proposed tasks improve the extraction of dialogue-based relations . speaker prediction captures the characteristics of speakerrelated entities . the trigger words prediction provides supportive contexts for relations between arguments . |
A Unified Linear-Time Framework for Sentence-Level Discourse Parsing (P19-1)
Copied to clipboard
| Challenge: | a new neural framework for sentence-level discourse analysis is proposed . a discourse segmenter and a parser are based on pointer networks and operate in linear time . |
| Approach: | They propose a neural framework for sentence-level discourse analysis in accordance with Rhetorical Structure Theory . they use a discourse segmenter and a parser to construct a discursive tree in a top-down fashion . |
| Outcome: | The proposed framework surpasses previous approaches on both tasks and human agreement on both. |
A Fair and In-Depth Evaluation of Existing End-to-End Entity Linking Systems (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations of entity linking systems often lack detailed error analysis or a closer look at the results. |
| Approach: | They evaluate existing entity linking systems and propose two new benchmarks . they characterize their strengths and weaknesses and report on reproducibility aspects . |
| Outcome: | The evaluations of existing system have strong biases and artifacts . they characterize their strengths and weaknesses and report on reproducibility aspects . |
AVA: an Automatic eValuation Approach for Question Answering Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | AVA is an automatic evaluation approach for question answering . it uses transformer-based language models to encode question, answer, and reference texts . |
| Approach: | They propose an automatic evaluation approach for Question Answering that uses Transformer-based language models to encode question, answer, and reference texts. |
| Outcome: | AVA can estimate system Accuracy with an error lower than 7% at 95% confidence level . the proposed approach achieves 74.7% F1 score in predicting human judgment for single answers . |
Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Question Answering (QA) is a field of increasing demand due to the availability of information online. |
| Approach: | They propose an unsupervised approach to training QA models with generated pseudo-training data by applying a simple template on a related sentence rather than the original context sentence. |
| Outcome: | The proposed approach improves the performance of a QA model on generated pseudo-training data. |
Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing report generation systems suffer from incomplete and inconsistent generation, despite achieving high performance on natural language metrics such as CIDEr and BLEU. |
| Approach: | They propose two new rewards that encourage the generation of factually complete and consistent radiology reports by using an existing semantic equivalence metric. |
| Outcome: | The proposed system significantly improves the F1 score of a clinical information extraction performance on two open radiology report datasets. |
Simple Questions Generate Named Entity Recognition Datasets (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent named entity recognition models rely on human-annotated datasets . however, in-domain dictionaries and sentences are often unavailable or expensive to construct for many entity types. |
| Approach: | They propose an ask-to-generate approach which automatically generates NER datasets by asking natural language questions to an open-domain question answering system. |
| Outcome: | The proposed model outperforms the previous best model by 19.5 F1 score on six benchmarks and achieves state-of-the-art performance. |
Document-level Entity-based Extraction as Template Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Document-level entity-based extraction (EE) tasks extract entity-centric information from unstructured text across multiple sentences. |
| Approach: | They propose a generative framework for two document-level EE tasks: role-filler entity extraction (RE) and relation extraction ( RE). |
| Outcome: | The proposed framework captures cross-entity dependencies and avoids exponential computation complexity of identifying N-ary relations. |
Lifelong Event Detection with Knowledge Transfer (2021.emnlp-main)
Copied to clipboard
| Challenge: | Traditional supervised Information Extraction (IE) methods can extract structured knowledge elements from unstructured data, but it is limited to a pre-defined target ontology. |
| Approach: | They propose a new lifelong event detection framework that is generalizable to other IE tasks and updates old knowledge with new event types’ mentions using a self-training loss. |
| Outcome: | The proposed framework outperforms baselines with a 5.1% gain in the F1 score and can boost the F2 score for over 30% on some new long-tail rare event types with few training instances. |
Late Fusion of Transformers for Sentiment Analysis of Code-Switched Data (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Code-switching is a common phenomenon in multilingual communities . however, sentiment analysis of code-switch data is elusive and unexplored . |
| Approach: | They propose to combine two transformers using logits of their output and feed them to a neural network for sentiment analysis. |
| Outcome: | The proposed system achieves an F1 score of 73.66% for English-Hi and 61.24% for English . it outperforms the best model reported for the GLUECoS benchmark dataset. |
Improving Accessibility of SCOTUS Opinions: A Benchmark Study and a New Dataset for Generic Heading Prediction and Specific Heading Generation (2025.coling-main)
Copied to clipboard
| Challenge: | SCOTUS opinions are notoriously long and use specialised language, making them laborious to read and understand. |
| Approach: | They propose generic and specific headings for each section to be trained automatically . they compare the performance of different systems trained for each subtask . |
| Outcome: | The proposed system can achieve a score of 0.90% in predicting generic headings . the proposed system also achieves similar scores in generating specific headings. |
Training Question Answering Models From Synthetic Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on question and answer generation aims to improve question answering models given limited amount of labeled data. |
| Approach: | They synthesize questions and answers from a synthetic text corpus generated by an 8.3 billion parameter GPT-2 model and achieve 88.4 Exact Match (EM) and 93.9 F1 score on the SQuAD1.1 dev set. |
| Outcome: | The proposed model achieves higher accuracy than the SQUAD1.1 training set questions using synthetic questions and answers than the training set question. |
Causal Discovery Inspired Unsupervised Domain Adaptation for Emotion-Cause Pair Extraction (2024.findings-emnlp)
Copied to clipboard
Yuncheng Hua, Yujin Huang, Shuo Huang, Tao Feng, Lizhen Qu, Christopher Bain, Richard Bassed, Reza Haf
| Challenge: | Emotion-cause pair extraction is a task that aims to extract emotions and the events causing such emotions. |
| Approach: | They propose a deep latent model which captures the underlying latent structures of data and utilizes the easily transferable knowledge of emotions as the bridge to link the distributions of events in different domains. |
| Outcome: | The proposed model outperforms the strongest baseline by approximately 11.05% on a Chinese benchmark and 2.45% on an English benchmark in terms of weighted-average F1 score. |
PhotoChat: A Human-Human Dialogue Dataset With Photo Sharing Behavior For Joint Image-Text Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | PhotoChat contains 12k dialogues, each of which is paired with a user photo that is shared during the conversation. |
| Approach: | They propose to use PhotoChat to facilitate research on image-text modeling by combining a photo-sharing intent prediction task and a picture retrieval task to retrieve the most relevant photo according to the dialogue context. |
| Outcome: | The proposed tasks achieve 10.4% recall@1 and 58.1% F1 scores, indicating that the proposed dataset presents interesting yet challenging real-world problems. |
Introducing a Large-Scale Dataset for Vietnamese POS Tagging on Conversational Texts (2020.lrec-1)
Copied to clipboard
| Challenge: | POS taggers are trained on informal texts which contain many informal inputs such as acronyms, abbreviations, out-of-vocabulary words, etc. |
| Approach: | They propose a large-scale human-labeled dataset for the Vietnamese POS tagging task on conversational texts and develop an annotation guideline to manually annotate 16.310K sentences using this guideline. |
| Outcome: | The proposed tagging scheme achieved 93.36% accuracy score and higher than the model with handcrafted features and fine-tuning BERT. |
Using Sentence-level Classification Helps Entity Extraction from Material Science Literature (2022.lrec-1)
Copied to clipboard
| Challenge: | Material Science research articles are a rich source of information about entities related to material science. |
| Approach: | They propose to use a sentence-level classifier to identify sentences containing at least one entity mention . they then apply the information extraction models only on the filtered sentences to extract various entities of interest. |
| Outcome: | The proposed model improves the F1 score by more than 4% . the proposed model removes redundant sentences from the articles that contain informative entities . |
Rethinking Negative Sampling for Handling Missing Entity Annotations (2022.acl-long)
Copied to clipboard
| Challenge: | Empirical studies show low missampling rate and high uncertainty are both essential for achieving promising performances with negative sampling. |
| Approach: | They propose an adaptive and weighted sampling distribution that further improves negative sampling by introducing missampling and uncertainty concepts. |
| Outcome: | The proposed approach improves on synthetic and well-annotated datasets in terms of F1 score and loss convergence. |
Benchmarking Diverse-Modal Entity Linking with Generative Models (2023.findings-acl)
Copied to clipboard
Sijia Wang, Alexander Hanbo Li, Henghui Zhu, Sheng Zhang, Pramuditha Perera, Chung-Wei Hang, Jie Ma, William Yang Wang, Zhiguo Wang, Vittorio Castelli, Bing Xiang, Patrick Ng
| Challenge: | Existing models for diverse-mode entity linking (EL) work well on per modality configurations, but it is more challenging to design a unified model for diverse modality. |
| Approach: | They propose a generative diverse-modal model that integrates text, image and table . they propose combining a multimodal encoder-decoder paradigm with a fine-tuning GDMM . |
| Outcome: | The proposed model outperforms state-of-the-art models by 8.51 F1 on average for diverse-modal EL. |
IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-domain QA tasks focus on questions whose answer can be deduced directly from global factual knowledge. |
| Approach: | They propose a dataset where each question is based on a counterfactual presupposition via an "if" clause. |
| Outcome: | The IfQA dataset contains 3,800 questions that were annotated by crowdworkers on relevant Wikipedia passages. |
Exploring Pre-trained Language Models for Event Extraction and Generation (P19-1)
Copied to clipboard
| Challenge: | Existing methods to extract event data are laborious to create and limited in size. |
| Approach: | They propose an event extraction model to overcome the roles overlap problem by separating the argument prediction in terms of roles. |
| Outcome: | The proposed method surpasses existing methods on the ACE2005 dataset and improves on the previous methods. |
Learning Latent Structures for Cross Action Phrase Relations in Wet Lab Protocols (2021.acl-long)
Copied to clipboard
| Challenge: | Lack of careful documentation has led to reproducibility crisis in biosciences . wet laboratory protocols are written in natural language and are unclear, composed of disconnected and distant parts and built upon implicit information that were referenced earlier or omitted entirely. |
| Approach: | They propose a model that extracts all action relationships across multiple sentences to generate a MSTG for a given protocol . they propose extending annotations in the WLP corpus for inter-sentence relations and implicit arguments to improve the model's temporal and causal relations. |
| Outcome: | The proposed model achieves an F1 score of 54.53% for temporal and causal relations in protocols from the corpus WLP-MSTG. |
SmartSpanNER: Making SpanNER Robust in Low Resource Scenarios (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is one of the most fundamental tasks in natural language processing. |
| Approach: | They propose a method which introduces a Named Entity Head (NEH) prediction task to SpanNER and performs multi-task learning together with task of span classification. |
| Outcome: | The proposed method improves the robustness of SpanNER in low resource scenarios on the CoNLL03, Few-NERD, GENIA and ACE05 benchmark datasets. |
Backdooring Neural Code Search (2023.acl-long)
Copied to clipboard
| Challenge: | Neural code search models are used to find code snippets from online repositories . however, their security aspect is rarely studied . |
| Approach: | They propose to use off-the-shelf code snippets from online repositories to find desired code . they propose to inject a backdoor into neural code search models which return buggy code if attacker modifies one variable/function name . |
| Outcome: | The proposed attack outperforms baselines on two neural code search models by 60%. |
PromptRank: Unsupervised Keyphrase Extraction Using Prompt (2023.acl-long)
Copied to clipboard
| Challenge: | Existing keyphrase extraction methods struggle with document and candidate length discrepancies or fail to fully utilize the pre-trained language model without further fine-tuning. |
| Approach: | They propose an unsupervised keyphrase extraction approach that uses a pre-trained language model to rank candidates based on document embeddings. |
| Outcome: | The proposed approach outperforms the existing keyphrase extraction approach on six benchmarks. |
Creating a Dataset for Named Entity Recognition in the Archaeology Domain (2020.lrec-1)
Copied to clipboard
| Challenge: | Currently, there is no way to find 'by-catch', single finds of a different type, in the metadata of excavation reports. |
| Approach: | They propose to train NER classifiers on Dutch excavation reports to help archaeologists find structured information in archaic documents. |
| Outcome: | The proposed dataset contains 31k annotations between six entity types (artefact, time period, place, context, species & material). |
Named Entity Recognition in Estonian 19th Century Parish Court Records (2022.lrec-1)
Copied to clipboard
| Challenge: | a corpus of 19th century Parish Court records annotated for named entities (NE) in Estonian is a valuable resource for historians, linguists and the public at large. |
| Approach: | They propose to annotate a corpus of Estonian Parish Court records annotated for named entities (NE) and report on named entity recognition experiments using this corpus. |
| Outcome: | The proposed model achieves microaverage F1 score of 93.6, comparable to state-of-the-art NER performance on the contemporary Estonian. |
Confront Insider Threat: Precise Anomaly Detection in Behavior Logs Based on LLM Fine-Tuning (2025.coling-main)
Copied to clipboard
| Challenge: | Current methods for insider threat detection suffer from low precision and information loss . a novel approach to detect insider threats is needed to improve accuracy . |
| Approach: | They propose a precise anomaly detection solution based on Large Language Model (LLM) fine-tuning . they represent user behavior in natural language and implement a threat tracing mechanism . |
| Outcome: | The proposed solution achieves an F1 score of 0.8941 on the CERT v6.2 dataset . |
IELM: An Open Information Extraction Benchmark for Pre-Trained Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show pre-trained LMs store linguistic and relational knowledge . pre-training LM models can answer "fill-in-the-blank" questions based on pre-defined relations . |
| Approach: | They propose an open information extraction benchmark for pre-trained language models . they turn pre-trained LMs into zero-shot OIE systems to examine open relational information . |
| Outcome: | The proposed benchmark outperforms state-of-the-art methods on factual OIE datasets without training sets. |
Confusionset-guided Pointer Networks for Chinese Spelling Check (P19-1)
Copied to clipboard
| Challenge: | Existing methods to detect and fix errors in Chinese are limited due to context. |
| Approach: | They propose a Confusionset-guided pointer network for Chinese Spell Check task . they propose to use off-the-shelf confusionset to guide character generation . |
| Outcome: | The proposed model outperforms all competitor models on three human-annotated datasets. |
Generalizable Implicit Hate Speech Detection Using Contrastive Learning (2022.coling-1)
Copied to clipboard
| Challenge: | Hate speech detection is challenging when there are insufficient lexical cues. |
| Approach: | They propose a contrastive learning method that pulls an implication and its corresponding posts close in representation space. |
| Outcome: | The proposed method improves on BERT and HateBERT benchmarks on three implicit hate speech benchmarks. |
Naamapadam: A Large-Scale Named Entity Annotated Data for Indic Languages (2023.acl-long)
Copied to clipboard
Arnav Mhaske, Harshit Kedia, Sumanth Doddapaneni, Mitesh M. Khapra, Pratyush Kumar, Rudra Murthy, Anoop Kunchukuttan
| Challenge: | Named Entity Recognition (NER) is a fundamental task in natural language processing (NLP). |
| Approach: | They present the largest publicly available Named Entity Recognition dataset for the 11 major Indian languages from two language families. |
| Outcome: | The proposed dataset is the largest publicly available Named Entity Recognition (NER) dataset for the 11 major Indian languages from two language families. |
Predicting Prerequisite Relations for Unseen Concepts (2022.emnlp-main)
Copied to clipboard
| Challenge: | Concept prerequisite learning (CPL) is a task of building a concept graph by structuring open knowledge in prerequisite relations. |
| Approach: | They propose to use both content-based and graph-based models to build a concept graph by structuring open knowledge in prerequisite relations. |
| Outcome: | The proposed approach improves F1 scores by 10% on three public benchmarks. |
Split or Merge: Which is Better for Unsupervised RST Parsing? (D19-1)
Copied to clipboard
| Challenge: | Rhetorical Structure Theory (RST) parsers have been based on supervised learning approaches that require an annotated corpus of sufficient size and quality. |
| Approach: | They propose two unsupervised methods that build an optimal RST tree based on a dissimilarity score function for splitting a text span into smaller ones and a similarity score for merging two adjacent spans into a large one. |
| Outcome: | The proposed method achieves the best score on English and German RST treebanks, around 0.8 F1 score, close to the previous supervised parsers. |
A Semi-Markov Structured Support Vector Machine Model for High-Precision Named Entity Recognition (P19-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is the backbone of many NLP solutions. |
| Approach: | They propose a neural semi-Markov structured support vector machine model that controls the precision-recall trade-off by assigning weights to different types of errors in the loss-augmented inference during training. |
| Outcome: | The proposed model achieves better precision-recall trade-off at various precision levels. |
Designing Informative Metrics for Few-Shot Example Selection (2024.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) have shown remarkable few-shot learning capabilities when provided with properly formatted examples. |
| Approach: | They propose a complexity-based prompt selection approach for sequence tagging tasks that uses certain metrics to align the syntactico-semantic complexity of test sentences and examples. |
| Outcome: | The proposed approach achieves state-of-the-art performance on few-shot NER, with 5% improvement in F1 score. |
Improving Answer Selection and Answer Triggering using Hard Negatives (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to answer selection and answer triggering have been proposed. |
| Approach: | They propose to use hard negatives with a siamese network and a suitable loss function for answer selection and answer triggering. |
| Outcome: | The proposed model improves on InsuranceQA, SelQA, and an internal QA dataset by 2.3 points over previous baselines. |
PARADE: A New Dataset for Paraphrase Identification Requiring Computer Science Domain Knowledge (2020.emnlp-main)
Copied to clipboard
| Challenge: | Paraphrase identification requires specialized domain knowledge to perform . state-of-the-art neural models and non-expert human annotators have poor performance on PARADE . |
| Approach: | They propose a benchmark dataset called PARADE for paraphrase identification that requires specialized domain knowledge. |
| Outcome: | The proposed dataset shows state-of-the-art models and non-expert human annotators have poor performance on PARADE. |
CorefQA: Coreference Resolution as Query-based Span Prediction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing coreference resolution models suffer from mention proposal. |
| Approach: | They propose a query-based span prediction task that can retrieve mentions left out at the mention proposal stage. |
| Outcome: | The proposed model can retrieve mentions left out at the mention proposal stage and improve generalization capability using existing question answering datasets. |
Mental Disorder Classification via Temporal Representation of Text (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current methods for mental disorder prediction split data into chunks and use limited context length . mental health professionals lack the skills to diagnose and treat mental disorders . |
| Approach: | They propose a framework which compresses chronologically ordered social media posts into a series of numbers and uses this time variant representation for mental disorder classification. |
| Outcome: | The proposed framework outperforms existing models in depression, self-harm and anorexia . it also shows that the proposed framework can be used across domains . |
Question-Answering in a Low-resourced Language: Benchmark Dataset and Models for Tigrinya (2023.acl-long)
Copied to clipboard
| Challenge: | Question-Answering (QA) has seen significant advances in recent years, achieving near human-level performance over some benchmarks. |
| Approach: | They propose to use a native QA dataset for an East African language, Tigrinya, to build similar resources for related languages. |
| Outcome: | The proposed method is applicable to constructing similar resources for related languages. |
FinMath: Injecting a Tree-structured Solver for Question Answering over Financial Reports (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing models for answering complex questions require multiple-step numerical reasoning. |
| Approach: | They propose a framework that injects a tree-structured neural model into a model to perform multi-step numerical reasoning. |
| Outcome: | The proposed framework improves the previous best model by 8.5% absolute for Exact Match (EM) score and 6.1% absolute for numeracy-focused F1 score. |
RadQA: A Question Answering Dataset to Improve Comprehension of Radiology Reports (2022.lrec-1)
Copied to clipboard
| Challenge: | Question answering (QA) is an intuitive means to query text data. |
| Approach: | They propose a radiology question-answer-evidence-pair dataset with 3074 questions posed against radiology reports and annotated with their corresponding answer spans by physicians. |
| Outcome: | The proposed dataset has 3074 questions posed against radiology reports and annotated with their corresponding answer spans by physicians. |
Agentic Medical Knowledge Graphs Enhance Medical Question Answering: Bridging the Gap Between LLMs and Evolving Medical Knowledge (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models have greatly advanced medical Question Answering (QA) however, the rapid evolution of medical knowledge and manual updating of domain-specific resources can undermine reliability of these systems. |
| Approach: | AMG-RAG automates the construction and continuous updating of Medical Knowledge Graph (MKG) . afriq: rapid evolution of medical knowledge and manual updating can undermine reliability of LLMs . |
| Outcome: | AMG-RAG achieves an F1 score of 74.1% on MEDQA and an accuracy of 66.34% on medMCQA. |
Systematic Comparison of Neural Architectures and Training Approaches for Open Information Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Open information extraction (OIE) is a method for extracting facts from text in structured format . alternative formulations allow for longer tuples, but most work focuses on binary predicates only. |
| Approach: | They propose to extract facts from natural language text and represent them as structured triples . they compare different neural network architectures and training approaches . |
| Outcome: | The proposed approach improves the currently best models on the OIE16 benchmark by 0.421 F1 score and 0.420 AUC-PR . |
HAWK: Highlighting Entity-aware Knowledge for Alleviating Information Sparsity in Long Contexts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a problem of information sparsity in QA tasks is causing fragmentation of textual data . highlighting entity-AWare Knowledge (HAWK) framework can be used to address this problem . |
| Approach: | a framework is proposed to highlight key information in a context and structuralize it in an entity-aware manner. |
| Outcome: | a proposed framework improves QA tasks with long contexts by highlighting key information in a context . the framework achieves a 27.6-point F1 score increase and an average win rate of 76.75% . |
Is Multihop QA in DiRe Condition? Measuring and Reducing Disconnected Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models exploit dataset artifacts to produce correct answers without connecting information across multiple facts. |
| Approach: | They formalize disconnected reasoning across subsets of supporting facts to reduce disconnected reasoning . they propose an automatic transformation of existing datasets that reduces disconnected reasoning. |
| Outcome: | The proposed model-agnostic probe reduces disconnected reasoning in a reading comprehension setting. |
MGR: Multi-generator Based Rationalization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to explain NLP models have two key challenges: spurious correlation and degeneration. |
| Approach: | They propose a rationalization framework using a generator and a predictor to construct a self-explaining NLP model with spurious correlation and degeneration as key challenges. |
| Outcome: | The proposed method improves the F1 score by 20.9% compared to state-of-the-art methods. |
Soft Gazetteers for Low-Resource Named Entity Recognition (2020.acl-main)
Copied to clipboard
| Challenge: | Existing named entity recognition models use gazetteers to improve performance, but they are limited in coverage and do not exist in low-resource languages. |
| Approach: | They propose a method that integrates Wikipedia information into named entity models by cross-lingual entity linking. |
| Outcome: | The proposed method improves on four low-resource languages with Wikipedia . it incorporates available information from english knowledge bases into neural models . |
How Robust Are the QA Models for Hybrid Scientific Tabular Data? A Study Using Customized Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing tabular QA models are lacking in understanding their robustness on scientific information. |
| Approach: | They propose a dataset to assess the robustness of tabular QA models on scientific hybrid tabular data. |
| Outcome: | The proposed model performs well on scientific tables and text, while the best score is 0.462. |
Unsupervised Cross-lingual Representation Learning at Scale (2020.acl-main)
Copied to clipboard
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov
| Challenge: | Pretraining multilingual language models at scale leads to performance gains for cross-lingual transfer tasks. |
| Approach: | They present a transformer-based multilingual masked language model pre-trained on 100 languages . they show that pretraining multilingual models at scale leads to significant performance gains . |
| Outcome: | The proposed model outperforms multilingual BERT (mBERT) on cross-lingual benchmarks. |
A Corpus of Turkish Offensive Language on Social Media (2020.lrec-1)
Copied to clipboard
| Challenge: | Identifying abusive, offensive, aggressive or in general inappropriate language has recently attracted interest of researchers from academic as well as commercial institutions. |
| Approach: | They propose to classify Turkish offensive language corpus using state-of-the-art annotation methods . they find 19 % of tweets contain some type of offensive language . |
| Outcome: | The proposed corpus of Turkish offensive language is the first of its kind in the world . the results show that 19 % of the tweets contain some type of offensive language . |
Maximal Clique Based Non-Autoregressive Open Information Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Open Information Extraction (OpenIE) aims to discover textual facts from a given sentence. |
| Approach: | They propose a non-autoregressive framework that generates a fact graph and a graph with an edge linking two nodes that belong to the same fact. |
| Outcome: | The proposed framework outperforms current state-of-the-art methods on two benchmark datasets and significantly outperformed the existing ones. |
CURE: Controlled Unlearning for Robust Embeddings — Mitigating Conceptual Shortcuts in Pre-Trained Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models are susceptible to spurious, concept-driven correlations that impair robustness and fairness. |
| Approach: | They propose a framework that disentangles and suppresses conceptual shortcuts while preserving essential content information. |
| Outcome: | The proposed framework improves on IMDB and Yelp datasets with minimal computational overhead. |
Improving Implicit Discourse Relation Recognition with Semantics Confrontation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for implicit discourse relation recognition (IDRR) are unsatisfactory for the task. |
| Approach: | They propose a method that trains PLMs through two semantics enhancers to implicitly differentiate logical and general semantics. |
| Outcome: | The proposed method exceeds baseline by 3.81% F1 score on PDTB 2.0 dataset . it infers discourse logical relations without explicit connectives, but performance remains unsatisfactory . |
ZeroNER: Fueling Zero-Shot Named Entity Recognition via Entity Type Descriptions (2025.findings-acl)
Copied to clipboard
Alessio Cocchieri, Marcos Martínez Galindo, Giacomo Frisoni, Gianluca Moro, Claudio Sartori, Giuseppe Tagliavini
| Challenge: | Existing zero-shot learning methods rely on entity type names for generalization . current solutions require large datasets and prioritize a handful of commonly occurring types . |
| Approach: | They propose a description-driven framework that enhances hard zero-shot NER in low-resource settings. |
| Outcome: | The proposed framework outperforms existing models by up to 16% in the F1 score . it also surpasses baseline models that use type names alone . |
XLS-R fine-tuning on noisy word boundaries for unsupervised speech segmentation into words (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to segment speech into words is needed to overcome the lack of explicit word boundaries in the speech stream. |
| Approach: | They propose to fine-tune a self-supervised speech model to predict word boundaries . they use XLS-R to fine tune the models and infer new word boundary labels . |
| Outcome: | The proposed model outperforms existing models and sets a new state-of-the-art on five corpora with different languages. |
Adversarial Robustness for Large Language NER models using Disentanglement and Word Attributions (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) tasks are becoming more challenging due to the introduction of complex tagsets, which often leads to the failure of existing NER systems in accurately recognizing these entities. |
| Approach: | They propose a novel attack which relies on disentanglement and word attribution techniques to learn an embedding and identifying important words across both components. |
| Outcome: | The proposed approach improves the F1 score over the original LLM model by 8% and 18% on CoNLL-2003 and Ontonotes 5.0 datasets respectively. |
HalluMeasure: Fine-grained Hallucination Measurement Using Chain-of-Thought Reasoning (2024.emnlp-main)
Copied to clipboard
Shayan Ali Akbar, Md Mosharaf Hossain, Tess Wood, Si-Chi Chin, Erica Salinas, Victor Alvarez, Erwin Cornejo
| Challenge: | HalluMeasure is a new LLM-based hallucination detection mechanism that decomposes an LLM response into atomic claims and evaluates each claim against the provided reference context. |
| Approach: | They propose a new LLM-based hallucination detection mechanism that decomposes an LLM response into atomic claims and evaluates each atomic claim against the provided reference context. |
| Outcome: | The proposed model can detect 3 major categories of hallucinations and 10 more specific subtypes which help to identify reasons behind the hallucinian errors. |
AfroXLMR-Social: Adapting Pre-trained Language Models for African Languages Social Media Text (2025.findings-emnlp)
Copied to clipboard
Tadesse Destaw Belay, Israel Abebe Azime, Ibrahim Said Ahmad, David Ifeoluwa Adelani, Idris Abdulmumin, Abinew Ali Ayele, Shamsuddeen Hassan Muhammad, Seid Muhie Yimam
| Challenge: | Domain adaptive pre-training and task-adaptive pre- training (TAPT) are popular methods to reduce this bias for low-resource languages, but they have not been explored for African multilingual encoders. |
| Approach: | They propose a large-scale social media and news domain corpus for continual pre-training on African languages. |
| Outcome: | The proposed methods improve performance on three subjective tasks, including sentiment analysis, multi-label emotion, and hate speech classification, while TAPT improves performance on other related tasks. |
Dual Cache for Long Document Neural Coreference Resolution (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies show the effectiveness of cache-based neural coreference resolution models on long documents. |
| Approach: | They propose a hybrid cache that integrates two eviction policies to capture global and local entities separately and improves F1 score of coreference by 0.7 5.7pt. |
| Outcome: | The proposed model outperforms existing models on four benchmarks while saving up to 83% of inference time. |
Unveiling Narrative Reasoning Limits of Large Language Models with Trope in Movie Synopses (2024.findings-emnlp)
Copied to clipboard
Hung-Ting Su, Ya-Ching Hsu, Xudong Lin, Xiang-Qian Shi, Yulei Niu, Han-Yuan Hsu, Hung-yi Lee, Winston Hsu
| Challenge: | Large language models (LLMs) equipped with chain-of-thoughts (CoT) prompting have shown significant multi-step reasoning capabilities in factual content like mathematics, commonsense, and logic. |
| Approach: | They introduce a trope-wise querying approach to assess the abstract reasoning abilities of large language models (LLMs) and uncover their low performance. |
| Outcome: | The proposed approach boosts the F1 score by 11.8 points and also reduces the performance of the large language models (LLMs) it also shows that it can cause hallucinations in narrative content, reducing the performance. |
SMARTMiner: Extracting and Evaluating SMART Goals from Low-Resource Health Coaching Notes (2025.findings-emnlp)
Copied to clipboard
Iva Bojic, Qi Chwen Ong, Stephanie Hilary Xinyi Ma, Lin Ai, Zheng Liu, Ziwei Gong, Julia Hirschberg, Andy Hau Yan Ho, Andy W. H. Khong
| Challenge: | SMARTMiner extracts specific, measurable, attainable, relevant, time-bound (SMART) goals from unstructured health coaching notes. |
| Approach: | They propose a framework for extracting and evaluating specific, measurable, attainable, relevant, time-bound (SMART) goals from unstructured health coaching notes. |
| Outcome: | The framework extracts behavior change goal spans and categorizes their SMARTness. |
Cross-Modal Attribute Insertions for Assessing the Robustness of Vision-and-Language Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to model multimodal data do not leverage cross-modal information . augmenting input text using cross-module attribute insertions results in poor performance . |
| Approach: | They propose a multimodal deep learning approach that adds visual attributes to inputs to enhance model robustness. |
| Outcome: | The proposed approach is modular, controllable, and task-agnostic. |
Focus on the Core: Efficient Attention via Pruned Token Compression for Document Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained transformers suffer from a computationally expensive self-attention mechanism that interacts with all tokens, including those unfavorable to classification performance. |
| Approach: | They propose to integrate token pruning and token combining strategies to improve model performance and reduce computational demands. |
| Outcome: | Experiments with various datasets show that the proposed model performs better than baseline models, with the best improvement over the existing model. |
FactAlign: Long-form Factuality Alignment of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have demonstrated significant potential as the next-generation information access engines, but reliability is hindered by issues of hallucination and generating non-factual content. |
| Approach: | They propose a novel alignment framework that enhances the factuality of LLMs’ long-form responses while maintaining their helpfulness. |
| Outcome: | The proposed framework improves factuality of LLMs while maintaining helpfulness. |
Are you sure? Measuring models bias in content moderation through uncertainty (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Language Model-based classifiers perpetuate racial and social biases in content moderation . et al., j. n. d., and j neil, e. c. (2005) measure the fairness of content moderated models . |
| Approach: | They propose an unsupervised approach that benchmarks models on their uncertainty . they use uncertainty as a proxy to analyze the bias of 11 models against women and non-whites . |
| Outcome: | The proposed method analyzes the bias of 11 models against women and non-white annotators . it shows that some pre-trained models predict with high accuracy the labels coming from minority groups . |
ReMedi: Reasoner for Medical Clinical Prediction (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to predicting future clinical outcomes from EHRs focus on enhancing medical knowledge through distillation or RAG while relying on the model’s internal ability to interpret contextual information. |
| Approach: | They propose a framework for improving clinical outcome prediction from EHR using a sample regeneration mechanism that leverages ground-truth answers as hints to enhance reasoning. |
| Outcome: | Experiments on multiple EHR prediction tasks show significant gains of up to 19.9% over state-of-the-art baselines in terms of F1 score, underscoring ReMedi’s effectiveness in real-world clinical prediction. |
Monolingual Paraphrase Detection Corpus for Low Resource Pashto Language at Sentence Level (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing research on sentence-level paraphrase detection in Pashto has focused on English, but no work has been done on low-resource Pashtone. |
| Approach: | They propose to annotate sentences in Pashto to detect paraphrases . they will publicize a subset of 1,800 instances from their corpus, free from licensing issues. |
| Outcome: | The proposed corpus contains 6,727 sentences, encompassing 3,687 paraphrased and 3,040 non-paraphrased sentences. |
Calibrated Seq2seq Models for Efficient and Generalizable Ultra-fine Entity Typing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | CASENT predicts ultra-fine entities mentioned in text into types with calibrated confidence scores. |
| Approach: | They propose a model that predicts ultra-fine entities with calibrated confidence scores for entity typing. |
| Outcome: | The proposed model outperforms existing models in terms of F1 score and calibration error while achieving 50 times faster inference speed. |
MMAG: Multimodal Learning for Mucus Anomaly Grading in Nasal Endoscopy via Semantic Attribute Prompting (2025.emnlp-main)
Copied to clipboard
| Challenge: | Accurate grading of rhinitis severity relies heavily on the characterization of key secretions, notably clear nasal discharge (CND) and purulent nasal secretion (PUS). |
| Approach: | They propose a framework that integrates structured prompts with rank-aware vision-language modeling for joint detection and grading. |
| Outcome: | The proposed model improves AUC and F1 scores on CND and PUS datasets by 6.31% and 4.79%. |
HACo-Det: A Study Towards Fine-Grained Machine-Generated Text Detection under Human-AI Coauthoring (2025.acl-long)
Copied to clipboard
| Challenge: | Existing literature focuses on binary, document-level detection, neglecting texts composed jointly by human and LLM contributions. |
| Approach: | They propose to use a dataset to generate human-AI coauthored texts via an automatic pipeline with word-level attribution labels. |
| Outcome: | The proposed method can detect human-AI coauthored texts with a numeric AI ratio. |
PM3-KIE: A Probabilistic Multi-Task Meta-Model for Document Key Information Extraction (2025.findings-acl)
Copied to clipboard
| Challenge: | a probabilistic multi-task meta-model for key information extraction is proposed . entity-level models capture spatial and visual cues, while fine-grained token classification models capture logical dependencies. |
| Approach: | They propose a probabilistic multi-task meta-model that integrates fine-grained token classification and coarse-grain entity extraction models. |
| Outcome: | The proposed model outperforms state-of-the-art models and a stacked ensemble on two datasets. |
XDAC: XAI-Driven Detection and Attribution of LLM-Generated News Comments in Korean (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models generate human-like text, raising concerns about their misuse in creating deceptive content. |
| Approach: | They propose a framework for detecting LLM-generated comments in Korean news and introduce a XDAC framework that leverages explainable AI to uncover distinguishing linguistic patterns at token and character levels. |
| Outcome: | The proposed framework outperforms existing methods and achieves 98.5% F1 score in detection and 84.3% F1 in attribution. |
Encoding Spreadsheets for Large Language Models (2024.emnlp-main)
Copied to clipboard
Haoyu Dong, Jianbo Zhao, Yuzhang Tian, Junyu Xiong, Mengyu Zhou, Yun Lin, José Cambronero, Yeye He, Shi Han, Dongmei Zhang
| Challenge: | Spreadsheets are characterized by their extensive two-dimensional grids, flexible layouts, and varied formatting options, which pose significant challenges for large language models (LLMs). |
| Approach: | They propose a structural-anchor-based compression, inverse index translation, and data-format-aware aggregation module to compress spreadsheets effectively. |
| Outcome: | The proposed method outperforms the existing model in GPT4 and achieves a state-of-the-art 78.9% F1 score. |
Automatic sentence segmentation of clinical record narratives in real-world data (2024.emnlp-main)
Copied to clipboard
| Challenge: | Sentence segmentation is a linguistic task used as a pre-processing step in many NLP applications. |
| Approach: | They propose a sequence labeling classifier that predicts sentence spans using a dynamic sliding window based on the prediction of each input sequence. |
| Outcome: | The proposed method outperforms state-of-the-art systems on clinical notes and on five other datasets to assess its generalizability and performance. |
QA-based Event Start-Points Ordering for Clinical Temporal Relation Annotation (2024.lrec-main)
Copied to clipboard
| Challenge: | Temporal relation annotation in the clinical domain is crucial but challenging due to its workload and the medical expertise required. |
| Approach: | They propose an annotation method that integrates event start-points ordering and question-answering as the annotation format. |
| Outcome: | The proposed method achieves a 0.72 F1 score and enables collaboration among medical experts and non-experts. |
The Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that LLM-based agents struggle to perform in zero-shot scenarios. |
| Approach: | They propose a framework to quantify the behavior gap between AI agents and human experts . they propose to examine discrepancies in dialog acts, tool usage, and knowledge utilization . |
| Outcome: | The proposed framework measures the behavior gap between AI agents and human experts on task-oriented dialogs. |
Document-Level Event-Argument Data Augmentation for Challenging Role Types (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Event Argument Extraction (EAE) are not well-suited to a variety of real-world situations, including long documents and challenging role types. |
| Approach: | They propose two novel methods for generating document-level EAE samples using zero in-domain training data and validate their generalizability. |
| Outcome: | The proposed methods show significant performance increases in low-resource settings. |
Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification (2026.findings-acl)
Copied to clipboard
Yuxuan Wan, Tianqing Fang, Zaitang LI, Yintong Huo, Wenxuan Wang, Haitao Mi, Dong Yu, Michael R. Lyu
| Challenge: | Recent advances in Deep Research Agents (DRAs) are transforming automated knowledge discovery and problem-solving. |
| Approach: | They propose an inference-time scaling of verification wherein an agent self-improves at test time by evaluating its generated answers. |
| Outcome: | The proposed model outperforms vanilla agent-as-judge and LLM judge baselines by 12%–48% in meta-evaluation F1 score. |
Temporal Relation Extraction in Clinical Texts: A Span-based Graph Transformer Approach (2025.acl-long)
Copied to clipboard
| Challenge: | Temporal information extraction from unstructured text is challenging due to complex clinical language, long documents, and sparse annotations. |
| Approach: | They propose a method for extracting clinical events and their temporal relations from unstructured text using the I2B2 2012 Temporal Relations Challenge corpus. |
| Outcome: | The proposed method improves state-of-the-art temporal information extraction with 5.5% improvement in tempeval F1 score over previous best and 8.9% improvement on long-range relations. |
Schema-based Data Augmentation for Event Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing data augmentation methods rely on language models to train event extraction models. |
| Approach: | They propose a schema-based data augmentation method that utilizes event schemas to guide the data generation process. |
| Outcome: | The proposed method produces high-quality generated data and significantly improves model performance. |
Semantics-Aware Dual Graph Convolutional Networks for Argument Pair Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Argument pair extraction (APE) aims to extract interactive argument pairs from two separate passages. |
| Approach: | They propose to tackle the lexical and semantic relevance of arguments with a pre-trained Rouge-guided Transformer (ROT) by using a word graph and a gating mechanism to fuse two graphs. |
| Outcome: | The proposed approach achieves state-of-the-art on F1 score and significantly improves on existing best alternative. |
Refining Attention for Explainable and Noise-Robust Fact-Checking with Transformers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Conventional transformer-based models falter due to noise sensitivity and lack explainability . ATTUN is a transformer architecture designed to enhance model transparency and resilience to noise. |
| Approach: | They propose a transformer architecture that enhances model transparency and resilience to noise . ATTUN is a module that directly modifies attention weights . they validated their approach using fact-checking datasets based on their results . |
| Outcome: | The proposed model improves predictions and identify relevant sections of input data. |
Syntactic Control of Language Models by Posterior Inference (2025.findings-acl)
Copied to clipboard
| Challenge: | Syntactic control of generated text is crucial for many domain-specific applications of language models, where structural constraints, such as formality, grammatical correctness, or adherence to a given template, can significantly affect usability and readability. |
| Approach: | They propose to use posterior inference to enforce target constituency structure by sampling from a proposal distribution with a syntactic tagger to ensure that each generated token aligns with the desired syntakic structure. |
| Outcome: | The proposed method increases the F1 score from 12.31 (GPT2-large) and 35.33 (Llama3-8B) to about 93 in both cases without compromising the language model’s fluency. |
VerilogLAVD: LLM-Aided Pattern Generation for Verilog CWE Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Existing static analysis tools focus on functional correctness and depend heavily on manual rules. |
| Approach: | They propose a framework that generates executable Traversal Detection Patterns (TDPs) to help detect hardware vulnerabilities. |
| Outcome: | The proposed framework improves the F1 score by 133% compared to LLM-based methods. |
LEMONADE: A Large Multilingual Expert-Annotated Abstractive Event Dataset for the Real World (2025.findings-acl)
Copied to clipboard
Sina Semnani, Pingyue Zhang, Wanyue Zhai, Haozhuo Li, Ryan Beauchamp, Trey Billing, Katayoun Kishi, Manling Li, Monica Lam
| Challenge: | Using a partially reannotated subset of the Armed Conflict Location & Event Data, we analyze 39,786 conflict events across 20 languages and 171 countries. |
| Approach: | They propose a large-scale conflict event dataset with extensive coverage of region-specific entities. |
| Outcome: | The proposed method detects event arguments and entities through holistic document understanding and normalizes them across the multilingual dataset. |
3MDBench: Medical Multimodal Multi-agent Dialogue Benchmark (2025.emnlp-main)
Copied to clipboard
Ivan Sviridov, Amina Miftakhova, Tereshchenko Artemiy Vladimirovich, Galina Zubkova, Pavel Blinov, Andrey Savchenko
| Challenge: | Large Vision-Language Models (LVLMs) are being explored in medicine but their ability to conduct complex real-world telemedicine consultations remains underexplored. |
| Approach: | They propose to use large vision-language models to conduct telemedicine consultations using a framework that simulates patient variability and evaluates diagnostic accuracy and dialogue quality via Assessor Agent. |
| Outcome: | The proposed framework compares diagnostic strategies for open and closed-source LVLMs and shows that multimodal dialogue improves F1 score by 6.5% over non-dialogue settings. |
Query-driven Document-level Scientific Evidence Extraction from Biomedical Studies (2025.acl-long)
Copied to clipboard
Massimiliano Pronesti, Joao H Bettencourt-Silva, Paul Flanagan, Alessandra Pascale, Oisín Redmond, Anya Belz, Yufang Hou
| Challenge: | Systematic reviews are widely regarded as the gold standard in evidence-based medicine, heavily influencing medical decisions made by doctors, health authorities, and patients. |
| Approach: | They propose a retrieval-augmented generation framework to tackle the unique challenges of evidence extraction by leveraging forest plots from Cochrane systematic reviews. |
| Outcome: | The proposed framework outperforms existing methods by up to 10.3% in the F1 score on this task. |
TED-EL: A Corpus for Speech Entity Linking (2024.lrec-main)
Copied to clipboard
| Challenge: | Current entity linking tasks rely on textual information, but entities usually exist in textual, audio, and visual contexts in real-world data such as social media and video websites. |
| Approach: | They propose a speech entity linking task to recognize mentions from speech and link them to entities in knowledge bases. |
| Outcome: | The proposed model outperforms the existing models on the TED-EL dataset, scoring an F1 score of 60.68%. |
ODASim: Ordered, Distinctive and Absolute Semantic Similarity for Code Explanation Evaluation (2026.findings-acl)
Copied to clipboard
Prince Kumar, Vitobha Munigala, Jaydeep Sen, Ashish Mittal, Vishwajeet Kumar, Srikanth G. Tamilselvam
| Challenge: | Existing methods for code explanations fail to distinguish correct from partially or fully incorrect explanations and their similarity scores are poorly calibrated. |
| Approach: | They propose a model-agnostic graded fine-tuning framework that learns calibrated similarity representations between code and explanations to support fine-grained supervision and evaluation. |
| Outcome: | The proposed framework improves F1 score and ECE scores on two embedding models and reduces expected calibration error. |
METAL: A Multi-Agent Framework for Chart Generation with Test-Time Scaling (2025.acl-long)
Copied to clipboard
| Challenge: | Chart generation requires strong visual design skills and precise coding capabilities that embed the desired visual properties into code. |
| Approach: | They propose a vision-language model-based multi-agent framework for effective automatic chart generation. |
| Outcome: | The proposed framework achieves a 5.2% improvement in the F1 score over the current best chart generation task. |
ETF: An Entity Tracing Framework for Hallucination Detection in Code Summaries (2025.acl-long)
Copied to clipboard
Kishan Maharaj, Vitobha Munigala, Srikanth G. Tamilselvam, Prince Kumar, Sayandeep Sen, Palani Kodeswaran, Abhijit Mishra, Pushpak Bhattacharyya
| Challenge: | Recent advances in large language models have significantly enhanced their ability to understand both natural language and code, but are prone to hallucinations. |
| Approach: | They propose a first-of-its-kind dataset, CodeSumEval, with 10K samples, curated specifically for hallucination detection in code summarisation. |
| Outcome: | The proposed framework has a 73% F1 score and is curated specifically for detection of hallucinations in code summarisation. |
Enhancing Study-Level Inference from Clinical Trial Papers via Reinforcement Learning-Based Numeric Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has framed this task as a textual inference task by retrieving relevant content fragments and inferring conclusions from them. |
| Approach: | They propose to extract structured numerical evidence and apply domain knowledge informed logic to derive outcome-specific conclusions. |
| Outcome: | The proposed approach outperforms general-purpose LLMs of over 400B parameters and achieves a 21% improvement in F1 score over retrieval-based systems. |
Spatial Layouts in News Homepages Capture Human Preferences (2025.emnlp-main)
Copied to clipboard
| Challenge: | Homepage layouts are daily and manually curated by expert human news editors, and serve as a proxy for information prioritization. |
| Approach: | They propose a pairwise preference model to capture ranked preferences between news items . they use a dataset of 3,000 news homepages captured twice daily over a five-year period . |
| Outcome: | The proposed model infers editorial preference with over 0.7 F1 score (based on human trials) it also ranks local city council policies passed over ten years in San Francisco, evaluating their "newsworthiness". |
Fine-Grained Detection of Context-Grounded Hallucinations Using LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing representations of hallucinations limit the types of errors that can be expressed, so we propose a new representation based on free-form textual descriptions, capturing the full range of possible errors. |
| Approach: | They propose a benchmark for localizing hallucinations using LLMs with a human annotation of over 1,000 examples and a protocol to verify its quality in a humans evaluation. |
| Outcome: | The proposed representation captures the full range of possible errors, and the best model achieves an F1 score of 0.67. |
GerAV: Towards New Heights in German Authorship Verification using Fine-Tuned LLMs on a New Benchmark (2026.findings-acl)
Copied to clipboard
| Challenge: | Authorship verification (AV) is a task of determining whether two texts were written by the same author. |
| Approach: | They propose a benchmark for German AV comprising over 400k labeled text pairs. |
| Outcome: | The proposed model outperforms baselines and state-of-the-art models by 0.09 and surpasses GPT-5 in a zero-shot setting by 0.08. |
Can LLM Agents Simulate Multi-Turn Human Behavior? Evidence from Real Online Customer Behavior Data (2026.acl-long)
Copied to clipboard
Yuxuan Lu, Jing Huang, Yan Han, Bingsheng Yao, Sisong Bei, Yaochen Xie, Yisi Sang, Qi He, Dakuo Wang
| Challenge: | Recent research shows that LLM Agents can generate “believable” human behaviors via prompt-only methods, leaving open questions of whether they can accurately generate step-by-step actions in multi-turn interaction tasks. |
| Approach: | They propose to use shopping data to evaluate LLMs' ability to accurately generate step-by-step actions in a multi-turn interaction task. |
| Outcome: | The proposed model achieves 17.26% action generation accuracy and 33.86% F1 score on final purchase prediction, representing improvements of 5.4% and 13.85% over baselines. |
Chain-of-Relations: Faithful and Efficient LLM Reasoning over Knowledge Graphs via Relation-Centric Exploration (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods adopt entity-centric exploration that incrementally constructs reasoning paths by selecting and connecting intermediate entities. |
| Approach: | They propose to use relation-centric exploration to construct reasoning paths by selecting and connecting intermediate entities and to reduce the dependence on entity completeness. |
| Outcome: | The proposed method outperforms baselines on three benchmark datasets in both F1 score and KG-grounded Rate. |